Commit Graph

666 Commits

Author SHA1 Message Date
Travis Geiselbrecht
ad6ef65a5a [arch][ops.h] force all of the fast routines to be inline
Also make sure each arch_ops.h always includes arch/ops.h at the top, to
make sure the declaration always appears in front of the definition.
2025-10-01 20:56:07 -07:00
Travis Geiselbrecht
04b88750b3 [debug] remove lib/debug and move to the top/ module
Most of the functions for this was declared in a top level lk/ include
space, so go ahead and move it there.

A few exceptions:
- Moved spin() over to platform/time.h and platform/time.c since the
function more logically belongs to platform/time.h.  Any users of
spin() will need to update their headers to include platform/time.h
instead.

- Renamed spin_cycles() to arm_cm_spin_cycles() and moved over into
arm/cm.h since it is currently defined in arch/arm-m and only used for
targets that implicitly are for arm-m.
2025-10-01 20:56:06 -07:00
Travis Geiselbrecht
e739abc490 [kernel] tweak a few thread apis to to take a const pointer
A bit of reformatting on some ARM code while was touching it.
2025-10-01 20:56:06 -07:00
Lei Wen
145e9a0d27 [arch] fix link script not include global array init
Using wild match init_array* to include global array init ctor

Before:

 10 .ctors             00000040 ffff0000001e6b80 00000000401e6b80 DATA
 11 .dtors             00000000 ffff0000001e6bc0 00000000401e6bc0 DATA
 12 .got               00000060 ffff0000001e6bc0 00000000401e6bc0 DATA
 13 .init_array.1      00000470 ffff0000001e6c20 00000000401e6c20
 14 .fini_array.1      00000470 ffff0000001e7090 00000000401e7090
 15 .dummy_post_data   00000000 ffff0000001e7500 00000000401e7500 DATA
 16 .bss               00009460 ffff0000001e8000 00000000401e8000 BSS

After:

 10 .ctors             000004b0 ffff0000001e6b80 00000000401e6b80 DATA
 11 .dtors             00000470 ffff0000001e7030 00000000401e7030
 12 .got               00000060 ffff0000001e74a0 00000000401e74a0 DATA
 13 .dummy_post_data   00000000 ffff0000001e7500 00000000401e7500 DATA
 14 .bss               00009460 ffff0000001e8000 00000000401e8000 BSS

Signed-off-by: Lei Wen <leiwen@asrmicro.com>
2025-09-29 22:40:41 -07:00
Travis Geiselbrecht
936ee8ac81 [arch][x86] start of an ioapic driver
Doesn't do much but provided the detection path for it and ability to
hold initialized state. The higher level platform code is going to need
to use it directly so will mostly just provide an api for access to it.

Moved ACPI sniffing back to just after the VM is initialized instead of
all the way into platform_init(). This should try to ensure that all
drivers that come up afterwards will have ioapics discovered in case
future development tries to enable and use them, kicking the machine out
of virtual-wire-mode.
2025-09-24 01:18:52 -07:00
Travis Geiselbrecht
5a17519e54 [arch][riscv] fix rounding issue with PAGE_SIZE on riscv64
Inadvertently added this bug where PAGE_SIZE type is 32bit which causes
truncating when using it to round a kernel address.
2025-09-23 23:05:15 -07:00
Travis Geiselbrecht
b7d69d8804 [arch][x86] handle the local apic of the boot cpu not being 0
I have a bulldozer machine here that curiously starts the APIC IDs for
the cpus at 16 and counts up.

This is a problem since the current code assumes that the boot cpu is 0,
and would try to start itself (apic id 16) later because it thought it
was the first secondary. Fix this by re-reading the APIC id on the boot
cpu and patching the percpu structure a bit into boot. Kinda a hack but
avoids having to detect the APIC, find the type of ID to read, etc.

Also means that practically speaking the system is using the full 32bit
APIC IDs if that feature is present, since now the local apic id is
entirely read from the local apic as it should be (if present).

Fixes #475
2025-09-22 20:57:30 -07:00
Travis Geiselbrecht
bdf2203fdc [arch][x86][lapic] a little bit of restructuring of some recent lapic code
Functionally equivalent, but refactor two copies of local apic
initialization code into a shared routine.
2025-09-21 21:53:35 -07:00
little fairy
d0eb5c464e [arch][x86] Read full 32bit apic id from x2apic msr if available (#465) 2025-09-21 21:04:00 -07:00
Travis Geiselbrecht
c48cfcb5d0 [arch][arm64] add support for 64k pages
Already worked, just needed to set up the address space size and create
a new project.
2025-08-31 21:48:09 -07:00
Travis Geiselbrecht
fff0f2a740 [arch][arm64] add support for 16k pages 2025-08-31 21:47:32 -07:00
Travis Geiselbrecht
cc9c3a053c [arch][mmu] clean up page size definitions in each arch's defines.h
No real functional change except how the smaller ARCH_DEFAULT_PAGE_SIZE
is now computed and set in defines.h instead of rules.mk for arch/arm to
be consistent with the other arch that has a large/small build (riscv).
2025-08-31 19:16:58 -07:00
Travis Geiselbrecht
be52909f49 [make][buildall] add ability to filter buildall by architecture
Clean up make targets list-arch and list-toolchain to be much faster and
work without needing to invoke the archtecture's arch rules.mk. This
should make it work on machines that do not have that particular
toolchain in the path.

This is setting up for using it in the github action script.
2025-07-18 21:51:50 -07:00
Travis Geiselbrecht
2d567e437d [kernel][vm] remove default KERNEL_ASPACE_BASE definition
Require the platform or arch to provide it. The default was pretty
obsolete, assuming a 32bit arch with a split 50/50 mapping. May as well
make it a forced arch thing so that it's known to be correct.

Had to define it for OR1K, it was already defined for all the other
(known) architectures.
2025-07-09 23:55:13 -07:00
Travis Geiselbrecht
37abbeb6db [arch][x86] remove lazy fpu save
For newer SMP and modern cpus, just doing a full fpu context switch
is simpler and cheaper than dealing with the complexity of lazy fpu
context switch logic.

Plus this old logic was broken in the recent SMP-ification of the x86
code base. A functional version of this would be far more complex.
2025-06-10 23:10:45 -07:00
Travis Geiselbrecht
e31b1ee768 [arch][m68k] make sure move %sp, <reg> doesn't use the 'a' registers
This particular instruction format doesn't take plain a registers, but
takes d and the rest of the <ea> formats.
2025-05-06 23:26:10 -07:00
Travis Geiselbrecht
0ac0911404 [arch][x86] save the translated kernel pgdir physical address
This keeps from needing to recompute it on every context switch back to
the kernel aspace.
2025-04-13 22:43:28 -07:00
Travis Geiselbrecht
a04776ba78 [arch][x86] add a few more feature bits
There's always more feature bits showing up on these things.
Rename a few to match the intel manual closer.
2025-04-11 01:21:12 -07:00
Travis Geiselbrecht
e970a6ea4b [arch][x86-64] remove extra segment register set in start.S 2025-04-10 22:25:38 -07:00
Travis Geiselbrecht
9d66069183 Merge from x86-smp: implement SMP for both 32 and 64bit x86
A large pile of changes to the PC platform and x86 architecture that
facilitate SMP support. Tested in both 64 and 32bit on qemu and real
hardware all the way back through i486.
2025-04-10 22:17:02 -07:00
Travis Geiselbrecht
c80efdc4ce [arch][x86] add missing pv.h file added a few commits back 2025-04-10 21:37:35 -07:00
Travis Geiselbrecht
c054ee89c2 [x86][mmu] only write to CR4 if necessary
In legacy builds it's possible to boot on a cpu that doesn't appear to
have CR4 implemented (Am586 to be precise), but there's no features
needed to set, so it seems that this was architecturally okay.
2025-04-10 00:45:41 -07:00
Travis Geiselbrecht
f52ef453fe [tests][mmu] update the file a bit for C++, add another test
Make sure a page appears to be unmapped after it is told to.
2025-04-08 23:49:47 -07:00
Travis Geiselbrecht
bc01491bc9 [arch][x86][mmu] disable SMAP, add PGE feature
32 and 64 bit:
- For now SMAP causes the mmu unit tests to fail, so disable.
- Make sure CR4.PGE is set if present.
- Make sure the rest of the system knows that user aspaces are available
  on 32bit.
2025-04-08 23:49:47 -07:00
Travis Geiselbrecht
699ec6344e [arch][x86][smp] few misc tweaks
- Skip cpus in the MADT table that are not enabled
- Bump count to 16 cpus
- Move the spurious interrupt vector to 0xff since it needs to end in
  0xf on <=P6.
2025-04-06 22:22:20 -07:00
Travis Geiselbrecht
a8bc048648 [arch][x86] tighten up x86 cpu bootstrap code
- Fix an assert in local apic code when not using x2apic and starting
  secondaries.
- Follow the spec a bit closer and wait up till a second for each
  secondary core to start.
2025-04-06 21:13:06 -07:00
Travis Geiselbrecht
3013662345 [arch][x86] move the KVM clock stuff to arch from platform
One can argue it more logically fits there, and eventually more KVM
features will arrive in this file that have nothing to do with timers.
2025-04-06 19:28:08 -07:00
Travis Geiselbrecht
71e795de19 [arch][x86] get SMP working on x86-32
- Added very basic user page table support (needed to bootstrap the
  secondary cpus)
- Added MP bootup code for 32bit.
2025-04-06 19:09:32 -07:00
Travis Geiselbrecht
8fdadd9b33 [arch][x86] implement basic spinlocks
-This fixes the instability, seems stable on x86-64.
2025-04-01 20:10:18 -07:00
Travis Geiselbrecht
5a520eca3e [arch][x86] start getting inter-processor-interrupts working
-Move the local apic driver to arch/x86
-Add routines to send IPIs between cpus

Something is unstable at the moment and the system crashes after a while
with random corruptions when using SMP.
2025-04-01 00:40:50 -07:00
Travis Geiselbrecht
2987f73d08 [platform][pc] add support for TSC based clock
-Detect if under KVM hypervisor and read tick rate or
-calibrate tick against PIT
2025-03-30 21:59:39 -07:00
Travis Geiselbrecht
6747010a5c [arch][arm64] restructure some start.S to allow KERNEL_VM and WITH_SMP
Currently only supports both set at the same time, but rearrange their
order such that theoretically both are pretty independent.

Somewhat untested in that there's no configuration for !KERNEL_VM and
WITH_SMP, but at least allows that config to be tested.
2025-01-11 17:12:16 -08:00
Travis Geiselbrecht
538f32e09d [arch][arm64] make sure SPSel is set to SP_ELx at boot
Just a safety check, has not been a problem in general, but pulled from
a similar trusty change.
2025-01-11 17:11:27 -08:00
Alex Richardson
d673d211eb riscv: add missing stdint.h include
While this is currently pulled in transitively, uint32_t is defined there
and we should explicitly include the appropriate header.
2025-01-07 22:28:07 -08:00
Alex Richardson
22ae6b6cea riscv: ensure arguments to __ASM_STR() are expanded first
Right now this does not cause any issues, but I had a local change where
missing expansion resulted in compilation errors when reading CSRs.
2025-01-07 22:28:07 -08:00
Alex Richardson
9241c78667 riscv: Add missing parens around macro argument
This triggered a out of range count overflow warning when
building for RV32 (calling the macro with `ticks >> 32`)
since the truncation happened before the shift.
2025-01-07 22:28:07 -08:00
Travis Geiselbrecht
d1a332891c [arch][x86] add x2apic mode to the local apic driver
Fill in some more x86 feature bits while at it.
2024-12-17 23:57:56 -08:00
Travis Geiselbrecht
6b89609887 WIP x86-64 SMP: get the 64bit secondaries fully started
Rearrange some of the cpu initialization code to be runnable on each cpu
as they come up. Complete the 64bit bootstrap mechanism and call into C
code.

Makes it as far as trying to reschedule via an IPI. Need to implement
local apic based IPI mechanism.
2024-12-13 00:21:16 -08:00
Travis Geiselbrecht
3ea007a237 [arch][x86] split the single GDT.S into two separate ones per subarch
It's getting too hard to maintain a single layout that works with both,
so go ahead and split it. Also redo the layout so it should be usable
with user space and syscall and sysenter instructions from either mode.
2024-12-12 22:23:50 -08:00
Travis Geiselbrecht
6538baea70 WIP x86-smp
add uspace mmu support for x86-64
trampoline x86-64 cpus to long mode and into the kernel aspace
2024-12-11 00:19:57 -08:00
Travis Geiselbrecht
1afb5d7a66 WIP x86 smp: start the framework for detecting and starting secondary cores 2024-12-06 23:40:26 -08:00
Travis Geiselbrecht
181796e843 WIP x86 get x86-32 working with a per-cpu gs: segment register for the kernel 2024-12-06 22:30:48 -08:00
Travis Geiselbrecht
902e2fcb8a WIP set up per cpu structures for x86-64
only on the boot cpu for now
2024-12-06 21:11:51 -08:00
Travis Geiselbrecht
fd79fccdde WIP x86 SMP 2024-12-06 00:03:48 -08:00
Travis Geiselbrecht
6f32a0f377 [arch][riscv] use newly discovered pseudo-instructions for load/stores
I hadn't noticed this before, but you can directly reference a global
variable in a load/store in assembly, which combines a lla + ld/sd into
a 2 instruction pair instead of 3 due to the 12 bit offset provided in
the load/store.
2024-11-27 21:53:29 -08:00
Travis Geiselbrecht
77eb84d152 [arch][x86] a few little tweaks while looking at some older code 2024-11-27 21:34:05 -08:00
Travis Geiselbrecht
2ca679aeca [arch][riscv][asm] use the call pseudoinstruction instead of jal
This fixes a problem if the text segment gets larger than ~1MB where the
raw jal instruction cannot reach. Using 'call' or 'tail' allows the
assembler to emit a 2 instruction sequence that the linker later
relaxes if it can.
2024-11-14 19:33:46 -08:00
Travis Geiselbrecht
52fa818e21 [arch][arm64] remove an unnecessary call to arm64_el3_to_el1
The existing arm64_elx_to_el1 already handles dropping the primary and
any secondary cpu down to el1 by the time this code path is reached.
2024-11-10 03:39:34 +00:00
Travis Geiselbrecht
6e39e5674c [arch][arm64] Make sure mpidr_el1 and mipr_el1 is configured
When dropping from EL2 (or EL3), load vmpidr_el2 and vpidr_el2 with the
correct values to make sure EL1 sees the 'real' mpidr_el1 and midr_el1.

Though in most cases they're already configured by whatever firmware ran
before, there's no actual guarantee that it is, and it may be full of
random garbage.
2024-11-10 03:31:23 +00:00
Travis Geiselbrecht
4102844048 [arm64][fpu] add fp arch extension around inline fpu asm
This quiets warnings on clang 18 about the missing fp arch extension
feature when using fp instructions.
2024-11-07 08:44:27 +00:00