146 Commits

Author SHA1 Message Date
Travis Geiselbrecht
e4d65228b5 [mp] restructure the sequence of how cpus are brought up
- Move a bit of the shared logic of secondary bootstrapping into a new
  function, lk_secondary_cpu_entry_early() which sets the current cpu
  pointer before calling the first half of the secondary LK_INIT
  routines.
- Create the per cpu idle threads on the main cpu instead of the
  secondary as they come up.
- Tweak all of the SMP capable architectures to use this new path.
- Move the top level mp routines into a separate file top/mp.c
- A bit more correctly ifdef out more SMP code.
2025-10-12 19:47:33 -07:00
Travis Geiselbrecht
664bb17afa [ubsan] fix some bugs and warnings discovered by ubsan
- X86 cpuid feature list dump was using the wrong array and walking off
  the end of one.
- GICv2 code had a left shift by up to 31 of an integer. Needs to be
  unsigned.
- PLIC same as GIC code.
- fdtwalker code should be using a bytewise accessor based helper
  function for reading large integers out of an unaliged FDT.
- PCI BIOS32 search code could do a 32bit unaligned read of a string,
  switch to using memcmp.
2025-10-05 15:35:31 -07:00
Travis Geiselbrecht
ad6ef65a5a [arch][ops.h] force all of the fast routines to be inline
Also make sure each arch_ops.h always includes arch/ops.h at the top, to
make sure the declaration always appears in front of the definition.
2025-10-01 20:56:07 -07:00
Travis Geiselbrecht
e739abc490 [kernel] tweak a few thread apis to to take a const pointer
A bit of reformatting on some ARM code while was touching it.
2025-10-01 20:56:06 -07:00
Lei Wen
145e9a0d27 [arch] fix link script not include global array init
Using wild match init_array* to include global array init ctor

Before:

 10 .ctors             00000040 ffff0000001e6b80 00000000401e6b80 DATA
 11 .dtors             00000000 ffff0000001e6bc0 00000000401e6bc0 DATA
 12 .got               00000060 ffff0000001e6bc0 00000000401e6bc0 DATA
 13 .init_array.1      00000470 ffff0000001e6c20 00000000401e6c20
 14 .fini_array.1      00000470 ffff0000001e7090 00000000401e7090
 15 .dummy_post_data   00000000 ffff0000001e7500 00000000401e7500 DATA
 16 .bss               00009460 ffff0000001e8000 00000000401e8000 BSS

After:

 10 .ctors             000004b0 ffff0000001e6b80 00000000401e6b80 DATA
 11 .dtors             00000470 ffff0000001e7030 00000000401e7030
 12 .got               00000060 ffff0000001e74a0 00000000401e74a0 DATA
 13 .dummy_post_data   00000000 ffff0000001e7500 00000000401e7500 DATA
 14 .bss               00009460 ffff0000001e8000 00000000401e8000 BSS

Signed-off-by: Lei Wen <leiwen@asrmicro.com>
2025-09-29 22:40:41 -07:00
Travis Geiselbrecht
936ee8ac81 [arch][x86] start of an ioapic driver
Doesn't do much but provided the detection path for it and ability to
hold initialized state. The higher level platform code is going to need
to use it directly so will mostly just provide an api for access to it.

Moved ACPI sniffing back to just after the VM is initialized instead of
all the way into platform_init(). This should try to ensure that all
drivers that come up afterwards will have ioapics discovered in case
future development tries to enable and use them, kicking the machine out
of virtual-wire-mode.
2025-09-24 01:18:52 -07:00
Travis Geiselbrecht
b7d69d8804 [arch][x86] handle the local apic of the boot cpu not being 0
I have a bulldozer machine here that curiously starts the APIC IDs for
the cpus at 16 and counts up.

This is a problem since the current code assumes that the boot cpu is 0,
and would try to start itself (apic id 16) later because it thought it
was the first secondary. Fix this by re-reading the APIC id on the boot
cpu and patching the percpu structure a bit into boot. Kinda a hack but
avoids having to detect the APIC, find the type of ID to read, etc.

Also means that practically speaking the system is using the full 32bit
APIC IDs if that feature is present, since now the local apic id is
entirely read from the local apic as it should be (if present).

Fixes #475
2025-09-22 20:57:30 -07:00
Travis Geiselbrecht
bdf2203fdc [arch][x86][lapic] a little bit of restructuring of some recent lapic code
Functionally equivalent, but refactor two copies of local apic
initialization code into a shared routine.
2025-09-21 21:53:35 -07:00
little fairy
d0eb5c464e [arch][x86] Read full 32bit apic id from x2apic msr if available (#465) 2025-09-21 21:04:00 -07:00
Travis Geiselbrecht
cc9c3a053c [arch][mmu] clean up page size definitions in each arch's defines.h
No real functional change except how the smaller ARCH_DEFAULT_PAGE_SIZE
is now computed and set in defines.h instead of rules.mk for arch/arm to
be consistent with the other arch that has a large/small build (riscv).
2025-08-31 19:16:58 -07:00
Travis Geiselbrecht
be52909f49 [make][buildall] add ability to filter buildall by architecture
Clean up make targets list-arch and list-toolchain to be much faster and
work without needing to invoke the archtecture's arch rules.mk. This
should make it work on machines that do not have that particular
toolchain in the path.

This is setting up for using it in the github action script.
2025-07-18 21:51:50 -07:00
Travis Geiselbrecht
37abbeb6db [arch][x86] remove lazy fpu save
For newer SMP and modern cpus, just doing a full fpu context switch
is simpler and cheaper than dealing with the complexity of lazy fpu
context switch logic.

Plus this old logic was broken in the recent SMP-ification of the x86
code base. A functional version of this would be far more complex.
2025-06-10 23:10:45 -07:00
Travis Geiselbrecht
0ac0911404 [arch][x86] save the translated kernel pgdir physical address
This keeps from needing to recompute it on every context switch back to
the kernel aspace.
2025-04-13 22:43:28 -07:00
Travis Geiselbrecht
a04776ba78 [arch][x86] add a few more feature bits
There's always more feature bits showing up on these things.
Rename a few to match the intel manual closer.
2025-04-11 01:21:12 -07:00
Travis Geiselbrecht
e970a6ea4b [arch][x86-64] remove extra segment register set in start.S 2025-04-10 22:25:38 -07:00
Travis Geiselbrecht
c80efdc4ce [arch][x86] add missing pv.h file added a few commits back 2025-04-10 21:37:35 -07:00
Travis Geiselbrecht
c054ee89c2 [x86][mmu] only write to CR4 if necessary
In legacy builds it's possible to boot on a cpu that doesn't appear to
have CR4 implemented (Am586 to be precise), but there's no features
needed to set, so it seems that this was architecturally okay.
2025-04-10 00:45:41 -07:00
Travis Geiselbrecht
bc01491bc9 [arch][x86][mmu] disable SMAP, add PGE feature
32 and 64 bit:
- For now SMAP causes the mmu unit tests to fail, so disable.
- Make sure CR4.PGE is set if present.
- Make sure the rest of the system knows that user aspaces are available
  on 32bit.
2025-04-08 23:49:47 -07:00
Travis Geiselbrecht
699ec6344e [arch][x86][smp] few misc tweaks
- Skip cpus in the MADT table that are not enabled
- Bump count to 16 cpus
- Move the spurious interrupt vector to 0xff since it needs to end in
  0xf on <=P6.
2025-04-06 22:22:20 -07:00
Travis Geiselbrecht
a8bc048648 [arch][x86] tighten up x86 cpu bootstrap code
- Fix an assert in local apic code when not using x2apic and starting
  secondaries.
- Follow the spec a bit closer and wait up till a second for each
  secondary core to start.
2025-04-06 21:13:06 -07:00
Travis Geiselbrecht
3013662345 [arch][x86] move the KVM clock stuff to arch from platform
One can argue it more logically fits there, and eventually more KVM
features will arrive in this file that have nothing to do with timers.
2025-04-06 19:28:08 -07:00
Travis Geiselbrecht
71e795de19 [arch][x86] get SMP working on x86-32
- Added very basic user page table support (needed to bootstrap the
  secondary cpus)
- Added MP bootup code for 32bit.
2025-04-06 19:09:32 -07:00
Travis Geiselbrecht
8fdadd9b33 [arch][x86] implement basic spinlocks
-This fixes the instability, seems stable on x86-64.
2025-04-01 20:10:18 -07:00
Travis Geiselbrecht
5a520eca3e [arch][x86] start getting inter-processor-interrupts working
-Move the local apic driver to arch/x86
-Add routines to send IPIs between cpus

Something is unstable at the moment and the system crashes after a while
with random corruptions when using SMP.
2025-04-01 00:40:50 -07:00
Travis Geiselbrecht
2987f73d08 [platform][pc] add support for TSC based clock
-Detect if under KVM hypervisor and read tick rate or
-calibrate tick against PIT
2025-03-30 21:59:39 -07:00
Travis Geiselbrecht
d1a332891c [arch][x86] add x2apic mode to the local apic driver
Fill in some more x86 feature bits while at it.
2024-12-17 23:57:56 -08:00
Travis Geiselbrecht
6b89609887 WIP x86-64 SMP: get the 64bit secondaries fully started
Rearrange some of the cpu initialization code to be runnable on each cpu
as they come up. Complete the 64bit bootstrap mechanism and call into C
code.

Makes it as far as trying to reschedule via an IPI. Need to implement
local apic based IPI mechanism.
2024-12-13 00:21:16 -08:00
Travis Geiselbrecht
3ea007a237 [arch][x86] split the single GDT.S into two separate ones per subarch
It's getting too hard to maintain a single layout that works with both,
so go ahead and split it. Also redo the layout so it should be usable
with user space and syscall and sysenter instructions from either mode.
2024-12-12 22:23:50 -08:00
Travis Geiselbrecht
6538baea70 WIP x86-smp
add uspace mmu support for x86-64
trampoline x86-64 cpus to long mode and into the kernel aspace
2024-12-11 00:19:57 -08:00
Travis Geiselbrecht
1afb5d7a66 WIP x86 smp: start the framework for detecting and starting secondary cores 2024-12-06 23:40:26 -08:00
Travis Geiselbrecht
181796e843 WIP x86 get x86-32 working with a per-cpu gs: segment register for the kernel 2024-12-06 22:30:48 -08:00
Travis Geiselbrecht
902e2fcb8a WIP set up per cpu structures for x86-64
only on the boot cpu for now
2024-12-06 21:11:51 -08:00
Travis Geiselbrecht
fd79fccdde WIP x86 SMP 2024-12-06 00:03:48 -08:00
Travis Geiselbrecht
77eb84d152 [arch][x86] a few little tweaks while looking at some older code 2024-11-27 21:34:05 -08:00
Travis Geiselbrecht
86267ca23c [include][reg.h] define new mmio_read/write accessors
To work properly with some hypervisors on various architectures (ARM,
ARM64, x86), add global routines to allow access to MMIO registers via
architecturally defined accessors.

Add accessors for ARM, ARM64, and x86-32/64. Have the other arches
default to just using whatever the compiler emits.

Will need to generally move things off the legacy REG*() accessors
since they're really not safe going forward with what compilers emit.
2024-05-13 00:39:29 -07:00
Travis Geiselbrecht
60bee01621 [arch][x86] stub out the cache routines 2024-05-09 19:54:54 -07:00
Travis Geiselbrecht
339ff8995a [arch][barriers] add default memory barriers for all of the architectures
Most are pretty straightforward, but a few of the more esoteric
architectures just defaults are implemented.
2024-05-09 19:51:32 -07:00
Travis Geiselbrecht
a070819c46 [clang] fix another warning that clang doesn't like 2024-04-01 23:48:50 -07:00
Travis Geiselbrecht
14f430d5e8 [arch][x86][clang] fix clang error introduced with exception cleanup
Remove extra declaration of assembly label which clang does not like.
2024-04-01 23:19:35 -07:00
Travis Geiselbrecht
ab0e1fd3a1 [arch][x86][exceptions] clean up the exception handlers
For both 32 and 64bit x86, have each of the exception stubs which push a
few words and branch to the common isr routine be simply 16 byte aligned
to make it easy to calculate the offset from the main isr table. This
cleans up some complexity that was actually broken for interrupts >= 0x80.

Also:
-Switch alignment directives to .balign
-Expand the x86-32 exception table to a full 256
-Remove an extraneous define
-Make sure the IDT is 8 or 16 byte aligned
-Use END_DATA and END_FUNCTION in the exception and gdt asm files
2024-03-10 21:50:01 -07:00
Alex Richardson
e3a463e585 [x86][clang] Allow clang to evaluate isr_stub_len
The current code results in
`error: invalid reassignment of non-absolute variable 'isr_stub_start'`.
Use a numbered label instead (as that can be reassigned) and reference
the last occurrence using the b suffix.
2023-06-07 15:55:50 -07:00
Alex Richardson
c2a31bf8a6 [x86][clang] Make return mnemonic compatible with clang
GCC appears to accept the Intel retf syntax in AT&T mode but clang is
stricter and requires use of lret.
2023-06-07 15:55:50 -07:00
Pedro Falcato
67a5104d23 [arch][x86] Change the cache line size to 64 bytes
All remotely modern x86 CPUs use 64-byte cache lines. Change CACHE_LINE
to reflect this.
2023-05-20 17:46:24 -07:00
Travis Geiselbrecht
bcfad25587 [arch][x86][mmu] update how mmu code accesses physical pages
Move the mmu_initial_mapping from platform into arch/x86. For this
architecture the default mappings are basically hard coded in arch/x86
anyway, so move ownership of this data there, closer to where it's
actually initialized.
Update the 32 and 64bit paging code to properly use the paddr_to_kvaddr
and vice versa routines.
Update to allocate page tables directly from the pmm instead of the heap
(on 32bit code).
2022-11-02 23:48:49 -07:00
Travis Geiselbrecht
7742bd021b [arch][x86] general cleanup
Clean up the x86.h file a bit with how constants are defined
Switch rdtsc to builtin
Added all of the known bits for the main CR registers.
Move the invlpg macro over to the common header.
Update comments in start.S
2022-11-02 23:02:24 -07:00
Travis Geiselbrecht
7ddde308a3 [arch][x86][mmu] fix unmap routines to actually work
also additionally invlpg the address.
2022-11-02 02:09:46 -07:00
Travis Geiselbrecht
7017a8fff9 [arch][mmu] add another routine to query if the arch supports user aspaces
Trim the arch mmu unit tests accordingly.

Should probably switch this to a #define, but it's possible some of
these queries could be dynamically detected (XN for example). May
revisit at some point.
2022-10-23 22:40:00 -07:00
Travis Geiselbrecht
2e81bfd55f [arch][x86] remove an extra printf left in the x86 mmu code 2022-10-23 21:00:15 -07:00
Travis Geiselbrecht
08b3238952 [arch][x86] clean up of mmu code
-Large cleanup of both the 32 and 64bit mmu code. Still the same general
flow but tighten up usage of mixing physical addresses with virtual
addresses.
-Remove the PAE code path from 32bit mmu, which was unused.
2022-10-23 20:37:33 -07:00
Travis Geiselbrecht
d5451cc8e6 [arch][tests] add a way to query some arch mmu features
Allow asking the arch layer if it supports NX pages or NS pages.
Have the arch mmu test code test accordingly.
Also tweak the tests to pass on arm32 mmu, which does not precisely
match the return semantics of the rest of the mmu routines on map/unmap.
2022-10-21 00:00:49 -07:00