Fix/smp#83
Open
KyleMao2023 wants to merge 9 commits into
Open
Conversation
**机制**：全局 shootdown 要所有核 ACK 一个 IPI；任何核在 `SpinNoIrqLock` 临界区（关 SIE）里都收不到 IPI。若该核卡在临界区出不来（自旋等一把被 shootdown 等待链持有的锁），就成环 -> 硬楔死（Ctrl+C 也无效）。 | 子类 | 根因 | 修复 | 状态 | | :--- | :--- | :--- | :--- | | **heap-grow ABBA** | `grow_virtual` / `reclaim` 持 `KERNEL_HEAP_VIRTUAL_LOCK` 跨 `shootdown_global_quiet` | 预留后放锁、CAS 回退；shootdown 移出锁 | ✅ 根修（消环） | | **grow 全局 shootdown 风暴** | `map_heap_pages` 对全新映射发无谓的全局 shootdown | 改本核 `sfence.vma` | ✅ 根修（消除主要触发源） | | **timer 全任务扫描（我引入的回归）** | `warn_lost_runnable_tasks` 同时持 16 把 `processor/rq` 锁，且持 `process_inner` 跨 `task_inner` | 回退 timer 调用 | ✅ 回退消除 | | **exec 长持锁** | `init_user_stack_from_strings`（拷 args/envs，慢）在 `task_inner` 内 | 移出锁，两头各短暂取锁 | ✅（exec 内仍有 1 处慢串操作待查） | | **Ctrl+C 调试 dump 长持锁** | `debug_dump_pgrp_tasks` 持 process/task 锁做 `warn!`（格式化+UART），多核同时跑互抢 | 锁内只快照、锁外再 `warn!` | ✅ | --- | 子类 | 根因 | 修复 | 状态 | | :--- | :--- | :--- | :--- | | **丢 IRQ** | `peek_used()` 读 DMA 写的 used ring 只用 `Acquire`（不保证对设备 DMA 可见） -> `handle_irq` 在「ISR 空+used 假空」窗口返回 | 三处 peek 前加 `virtio_dma_rmb()`（`fence iorw, iorw`） | ✅ 根修 + 丢 IRQ 自愈兜底 | --- | 子类 | 根因 | 修复 | 状态 | | :--- | :--- | :--- | :--- | | **`wake_current` 孤儿** | `wake_running_or_queued_task` 把仍 current 的任务置 Runnable 但不入队，靠「所在核会继续跑它」的脆弱假设 | 加 `resched_hart(last_cpu)` + worker 单任务自愈 |⚠️ 缓解（根因 `trap_from_kernel` 末尾无 `schedule_if_needed` 未补） | | **快照脏读假阳性** | `block_worker_debug_snapshot` 在取 `task_inner` 锁前用 Relaxed 读 `on_cpu` -> 错位 | 移到锁内读 | ✅ | | **mutex 级联（下游）** | unlock 只 `wake_one`，被唤醒者成孤儿 -> 链断 | 依赖 C 的自愈 |⚠️ 软问题（Ctrl+C 可救） | TODO - 理论上不应该触发 `self-heal`，其原理有待考究 - `SpinNoIrqLock` 中减少慢操作只是实际操作上减少概率，没有理论上完全消除
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.
This PR primarily addresses SMP/interrupt-safety issues (notably around TLB shootdowns and long IRQ-disabled lock holds), while also expanding OS/user functionality for sockets, signals, file locking, and diagnostics.
Changes:
- Reduce system-wedge risks by shortening IRQ-disabled critical sections (exec/heap/scheduler dumps) and adding stall diagnostics/self-heal paths.
- Add missing/extended syscall support (sockopts,
rt_sigpending,MSG_NOSIGNAL, UDP/TCP sendmsg/recvmsg name handling). - Implement POSIX record locks (
fcntl), plus release-on-close/exec/exit integration; add tty winsize probing and a DNS probe user app.
Reviewed changes
Copilot reviewed 28 out of 28 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| user/src/syscall.rs | Adds setsockopt/getsockopt syscall numbers and user wrappers. |
| user/src/net.rs | Introduces socket option constants for timeouts. |
| user/src/lib.rs | Adds safe-ish convenience wrappers for setting sockopts from bytes/timeval. |
| user/src/bin/dns_probe.rs | New DNS UDP probe utility with optional recv timeout. |
| os/src/timer.rs | Adds periodic block-device stall warning call from timer interrupt. |
| os/src/task/task.rs | Adds per-task last scheduler op stamp + new WaitReason::FileLock. |
| os/src/task/process.rs | Releases POSIX locks on cloexec during exec; reduces IRQ-off lock hold during exec stack init. |
| os/src/task/mod.rs | Exports LastSchedOp; releases POSIX locks on process exit; avoids logging under locks in task dump. |
| os/src/syscall/signal.rs | Implements rt_sigpending(2). |
| os/src/syscall/net.rs | Allows MSG_NOSIGNAL; adds UDP/TCP support for sendmsg/recvmsg name handling. |
| os/src/syscall/mod.rs | Wires SYSCALL_RT_SIGPENDING into dispatcher. |
| os/src/syscall/fs.rs | Adds fcntl POSIX record locks and releases locks on close/close_range/dup2. |
| os/src/sched/runqueue.rs | Adds lost-runnable detection/self-heal + logging counters; refactors wake/enqueue paths. |
| os/src/sched/processor.rs | Makes current_ptr() available without feature gating. |
| os/src/net/udp.rs | Adjusts wildcard bind behavior to preserve external reachability. |
| os/src/mm/tlb_shootdown.rs | Adds stall-progress logging in shootdown ack wait loop. |
| os/src/mm/oom.rs | Removes warn_heap_state_lockfree. |
| os/src/mm/mod.rs | Stops re-exporting removed warn_heap_state_lockfree. |
| os/src/mm/memory_set.rs | Improves munmap teardown to avoid per-page VMA lookups; tweaks merge/flush behavior. |
| os/src/mm/heap_allocator.rs | Avoids deadlocks by dropping virtual-heap lock before mapping/unmapping; removes global shootdown for fresh heap mappings. |
| os/src/fs/tty.rs | Adds xterm-style winsize probing + SIGWINCH emission on resize. |
| os/src/fs/mod.rs | Implements POSIX record locks and conflict querying with wait queues. |
| os/src/drivers/virtio/mod.rs | Adds LoongArch DMA fences and a write-barrier helper. |
| os/src/drivers/block/virtio_blk.rs | Adds DMA-read fencing + extensive stall diagnostics counters and helpers. |
| os/src/drivers/block/mod.rs | Adds block-worker debug snapshot + timer-driven stall warnings and self-heal. |
| Makefile | Adds rootfs init stamps, LA arch validation, and reduces rootfs variant resync. |
| CosmOS-rootfs | Updates rootfs submodule pointer. |
| .vscode/settings.json | Adjusts rust-analyzer settings and expands watcher excludes. |
Comment on lines
+610
to
+633
| WinSizeReportParser::Esc { mut len, mut buf } => { | ||
| if len >= buf.len() { | ||
| state.winsize_parser = WinSizeReportParser::idle(); | ||
| state.winsize_probe_pending = false; | ||
| return WinSizeReportResult::Replay { buf, len }; | ||
| } | ||
| buf[len] = raw; | ||
| len += 1; | ||
| if raw == b'[' { | ||
| state.winsize_parser = WinSizeReportParser::Csi { | ||
| len, | ||
| buf, | ||
| field: 0, | ||
| value: 0, | ||
| kind: 0, | ||
| rows: 0, | ||
| cols: 0, | ||
| }; | ||
| WinSizeReportResult::Consumed | ||
| } else { | ||
| state.winsize_parser = WinSizeReportParser::idle(); | ||
| WinSizeReportResult::Replay { buf, len } | ||
| } | ||
| } |
Comment on lines
+513
to
+523
| for (pid, process) in processes { | ||
| let process_inner = process.inner_exclusive_access(); | ||
| let pgid = process_inner.cred.pgid; | ||
| let process_zombie = process_inner.is_zombie; | ||
| let exec_path = process_inner.exec_path.clone(); | ||
| for (tid, task) in process_inner.tasks.iter().enumerate() { | ||
| let Some(task) = task.as_ref() else { | ||
| continue; | ||
| }; | ||
| let task_ptr = Arc::as_ptr(task) as usize; | ||
| let task_inner = task.inner_exclusive_access(); |
Comment on lines
+546
to
+572
| warn!( | ||
| "[sched-inv][lost-runnable] reason={} count={} task={:#x} pid={} \ | ||
| tid={} pgid={} exec={} process_zombie={} wait={:?} last_cpu={} \ | ||
| policy={:?} on_cpu={} on_rq={} has_wq={} task_pending={:#x} \ | ||
| mask={:#x} resched={:?} currents={:?} rq_snapshot={:?} \ | ||
| last_sched_op={:?}", | ||
| reason, | ||
| count, | ||
| task_ptr, | ||
| pid, | ||
| tid, | ||
| pgid, | ||
| exec_path, | ||
| process_zombie, | ||
| task_inner.wait_reason, | ||
| task_inner.sched.last_cpu, | ||
| task_inner.sched.policy, | ||
| on_cpu, | ||
| task_inner.sched.on_rq, | ||
| task_inner.current_wq_handle.is_some(), | ||
| task_inner.pending_signals.bits(), | ||
| task_inner.signal_mask.bits(), | ||
| task_inner.sched.resched_reason, | ||
| current_ptrs, | ||
| rq_snapshot, | ||
| task_inner.last_sched_op, | ||
| ); |
Comment on lines
+529
to
+536
| if !matches!(task_inner.task_status, TaskStatus::Runnable) | ||
| || on_cpu | ||
| || task_inner.sched.on_rq | ||
| || current_ptrs.contains(&task_ptr) | ||
| || runnable_ptrs.contains(&task_ptr) | ||
| { | ||
| continue; | ||
| } |
Comment on lines
+1027
to
1033
| let mut removed_from = None; | ||
| for (hart, rq) in RUN_QUEUES.iter().enumerate() { | ||
| if rq.lock().remove_task(&task) { | ||
| removed_from = Some(hart); | ||
| break; | ||
| } | ||
| } |
Comment on lines
505
to
+508
| static ref FLOCK_TABLE: SpinNoIrqLock<Vec<FlockRecord>> = SpinNoIrqLock::new(Vec::new()); | ||
| static ref POSIX_LOCK_TABLE: SpinNoIrqLock<BTreeMap<(u64, u64), Arc<PosixLockState>>> = | ||
| SpinNoIrqLock::new(BTreeMap::new()); | ||
| } |
Comment on lines
+519
to
+530
| fn posix_lock_state(fs_id: u64, ino: u64, create: bool) -> Option<Arc<PosixLockState>> { | ||
| let mut table = POSIX_LOCK_TABLE.lock(); | ||
| if let Some(state) = table.get(&(fs_id, ino)) { | ||
| return Some(Arc::clone(state)); | ||
| } | ||
| if !create { | ||
| return None; | ||
| } | ||
| let state = Arc::new(PosixLockState::new()); | ||
| table.insert((fs_id, ino), Arc::clone(&state)); | ||
| Some(state) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.