Skip to content

Commit bf7eda0

Browse files
jaydee-coderdavide125
jaydee-coder
authored and
davide125
committed
SW-Hypervisor: Formatting fixes
- Force new lines by adding two space at the end of sentences. - Properly format code blocks that are part of bullet points by indenting them by 8 spaces as per mkdocs requirements. - Minor wording/punctuation fixes. - Visually de-cluttered some parts of the doc. - No actual content changes. Signed-off-by: jaydee-coder <[email protected]>
1 parent 53e27e5 commit bf7eda0

1 file changed

Lines changed: 95 additions & 54 deletions

File tree

‎docs/SW-Hypervisor.md‎

Lines changed: 95 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -7,90 +7,131 @@ You can run either a development kernel obtained from Apple, in which case you w
77
You can use either your existing macOS install, or alternatively install a second copy of macOS.
88

99
To install a second copy of macOS you will need to complete a couple of steps:
10-
1. Create a second Volume on your macOS partition:
11-
`diskutil apfs addVolume disk4 APFS macOSTest -mountpoint /Volumes/macOSTest` Change disk4 and volume name (i.e macOSTest) for your particular system/preferences. _Note: Don't make this a system role or it will mess with your existing system (no valid users in 1TR)_
12-
2. Download and install macOS. To download a specific version of macOS installer you can use the command
13-
`softwareupdate --fetch-full-installer --full-installer-version 12.3` substituting 12.3 for whichever version you require. The installer will be found in the Applications folder. Copy it out of here if you want to save it, otherwise it deletes itself once you have installed once.
1410

15-
Unfortunately, Apple's CDN only keeps the full-installer package for a limited number of version, and doesn't have 12.3 anymore. Note: we are now at firmware version 13.5, which is available normally. You don't need to install 12.3.
11+
1. Create a second Volume on your macOS partition:
12+
13+
diskutil apfs addVolume disk4 APFS macOSTest -mountpoint /Volumes/macOSTest
14+
15+
Change disk4 and volume name (i.e `macOSTest`) for your particular system/preferences.
16+
_Note: Don't make this a system role or it will mess with your existing system (no valid users in 1TR)_.
17+
18+
2. Download and install macOS. To download a specific version of macOS installer you can use the command:
19+
20+
softwareupdate --fetch-full-installer --full-installer-version 12.3
21+
22+
Substitute `12.3` for whichever version you require. The installer will be found in the Applications folder. Copy it out of here if you want to save it, otherwise it deletes itself once you have installed once.
23+
24+
Unfortunately, Apple's CDN only keeps the full-installer package for a limited number of version, and doesn't have 12.3 anymore.
25+
_Note: we are now at firmware version 13.5, which is available normally. You don't need to install 12.3._
1626

1727
### Using archived InstallAssistant.pkg
1828

19-
The Montery 12.3 InstallAssistant.pkg has been archived [here](https://archive.org/details/12.3-21-e-230-release), but trying to install by double-clicking installs an online version of the `Install macOS Monterey.app`, with a file size of about 40mb. When that run, that will install the latest version of macOS.
29+
The Montery 12.3 InstallAssistant.pkg has been archived [here](https://archive.org/details/12.3-21-e-230-release), but trying to install by double-clicking installs an online version of the `Install macOS Monterey.app`, with a file size of about 40MB. When that file is run, it will install the latest version of macOS. However, installing it via the command line appears to do the correct thing:
2030

21-
But installing it via the command line appears to do the correct thing: `sudo installer -pkg 12.3\ 21E230\ \(Release\).pkg -target /`
31+
sudo installer -pkg 12.3\ 21E230\ \(Release\).pkg -target /
2232

23-
Check that `Install macOS Monterey.app` in the applications folder is ~12GB.
33+
34+
Check that `Install macOS Monterey.app` in the `applications` folder is ~12GB.
2435

2536

2637
## Getting the macOS development kernel and creating the kernelcache
2738

28-
1. Create a macOS developer account (requires an icloud account.
29-
2. Download the Mac OS Kernel Debug Kit (KDK) from Apple: https://developer.apple.com/download/more/, it should match your Mac OS version.
30-
3. Install the KDK into Mac OS, the KDK will be installed to `/Library/Developer/KDKs/KDK_<MACOS_VERSION>_<KDK_VERSION>.kdk`
31-
4. Change to the kernels directory: `cd /Library/Developer/KDKs/KDK_<MACOS_VERSION>_<KDK_VERSION>.kdk/System/Library/Kernels`
39+
1. Create a macOS developer account (requires an iCloud account).
40+
2. Download the Mac OS Kernel Debug Kit (KDK) from Apple [here](https://developer.apple.com/download/more/). It should match your Mac OS version.
41+
3. Install the KDK into Mac OS. The KDK will be installed to `/Library/Developer/KDKs/KDK_<MACOS_VERSION>_<KDK_VERSION>.kdk`
42+
4. Change to the kernels directory:
43+
44+
cd /Library/Developer/KDKs/KDK_<MACOS_VERSION>_<KDK_VERSION>.kdk/System/Library/Kernels
45+
3246
5. Switch into the KDK folder and run the following command:
33-
```
34-
kmutil create -z -n boot -a arm64e -B ~/dev.kc.macho -V development \
35-
-k kernel.development.t8101 \
36-
-r /System/Library/Extensions/ \
37-
-r /System/Library/DriverExtensions \
38-
-x $(kmutil inspect -V release --no-header | grep -v "SEPHiber" | awk '{print " -b "$1; }')
39-
```
40-
`-B` designates the output file, our kernel cache is written to `dev.kc.macho` in the home directory
4147

42-
`-k` must match a kernel in the kernels directory
48+
kmutil create -z -n boot -a arm64e -B ~/dev.kc.macho -V development \
49+
-k kernel.development.t8101 \
50+
-r /System/Library/Extensions/ \
51+
-r /System/Library/DriverExtensions \
52+
-x $(kmutil inspect -V release --no-header | grep -v "SEPHiber" | awk '{print " -b "$1; }')
53+
54+
`-B` designates the output file, our kernel cache is written to `dev.kc.macho` in the home directory
55+
56+
`-k` must match a kernel in the kernels directory
4357

4458
## Preparing the macOS Volume by disabling security features
4559

46-
0. Set the macOS Volume as a default boot target
47-
1. Start into 1tr and start a terminal
48-
2. Disable most security feature in the boot policy: `bputil -nkcas`; use `diskutil info [disk name]` to get UUID
49-
3. Disable SIP (bputil resets it): `csrutil disable`
50-
4. install [m1n1](m1n1-User-Guide.md) as custom boot object
51-
```
52-
kmutil configure-boot -c build/m1n1.bin --raw --entry-point 2048 --lowest-virtual-address 0 -v /Volumes/macOSTest
53-
```
60+
0. Set the macOS Volume as a default boot target.
61+
1. Start into 1tr and start a terminal.
62+
2. Disable most security feature in the boot policy: `bputil -nkcas`; use `diskutil info [disk name]` to get UUID.
63+
3. Disable SIP (bputil resets it): `csrutil disable`.
64+
4. Install [m1n1](m1n1-User-Guide.md) as custom boot object:
65+
66+
kmutil configure-boot \
67+
-c build/m1n1.bin \
68+
--raw \
69+
--entry-point 2048 \
70+
--lowest-virtual-address 0 \
71+
-v /Volumes/macOSTest
5472

5573
## Starting the development kernel under the m1n1 hypervisor
5674

57-
1. Copy the kernelcache to your development machine
75+
1. Copy the kernelcache to your development machine.
5876
2. Copy the debug DWARF from `/Library/Developer/KDKs/KDK_<MACOS_VERSION>_<KDK_VERSION>.kdk/System/Library/Kernels/kernel.development.t8101.dSYM/Contents/Resources/DWARF/kernel.development.t8101`
59-
3. Run
60-
```python3 proxyclient/tools/run_guest.py -s <PATH_TO_DEBUG_DWARF> <PATH_TO_DEVELOPMENT_KERNEL_CACHE> -- "debug=0x14e serial=3 apcie=0xfffffffe -enable-kprintf-spam wdt=-1 clpc=0"```
61-
to start macOS with the m1n1 hypervisor.
77+
3. To start macOS with the m1n1 hypervisor, run:
78+
79+
python3 proxyclient/tools/run_guest.py \
80+
-s <PATH_TO_DEBUG_DWARF> \
81+
<PATH_TO_DEVELOPMENT_KERNEL_CACHE> \
82+
-- "debug=0x14e serial=3 apcie=0xfffffffe -enable-kprintf-spam wdt=-1 clpc=0"
6283

63-
Note: t8101 files(both kernel and dwarf symbols) are available starting with KDK for macOS 11.3. Marcan streams on booting macOs with hypervisor were done with 11.3. These notes have also been validated with macOS 11.5.2 and can get to login with wifi network on MacBookAir:
64-
* Kernel version: uname -a => Darwin MacBook-Air.home 20.6.0 Darwin Kernel Version 20.6.0: Wed Jun 23 00:26:27 PDT 2021; root:xnu-7195.141.2~5/RELEASE_ARM64_T8101 arm64
65-
* macOS version: sw_vers => ProductName: macOS / ProductVersion: 11.5.2 / BuildVersion: 20G95
84+
Note: t8101 files (both kernel and dwarf symbols) are available starting with KDK for macOS 11.3. Marcan streams on booting macOs with hypervisor were done with 11.3.
85+
These notes have also been validated with macOS 11.5.2 and can get to login with WiFi network on a MacBook Air:
86+
87+
- Kernel version:
88+
```
89+
$ uname -a
90+
> Darwin MacBook-Air.home 20.6.0 Darwin Kernel Version 20.6.0: Wed Jun 23 00:26:27 PDT 2021; root:xnu-7195.141.2~5/RELEASE_ARM64_T8101 arm64
91+
```
92+
- macOS version:
93+
```
94+
$ sw_vers
95+
> ProductName: macOS / ProductVersion: 11.5.2 / BuildVersion: 20G95
96+
```
6697

67-
If you see the apple logo(rainbow version) and not the progress bar, this means you probably had a macOS panic early in the boot process.
68-
This can come from a mismatch of macOS version between the kernel cache (kernel+extensions) and the macOS root fs.
69-
To figure out where the boot process is stuck, you can start a serial utility like minicom/picocom and the like, with 115200 baud rate (something like picocom -b 115200 /dev/ttyACM1).
70-
Be patient when you are booting, with one cpu and the hypervisor, depending on what you are tracing, it is slower than normal and it is expected.
98+
If you see the apple logo (rainbow version) and not the progress bar, this means you probably had a macOS panic early in the boot process. This can come from a mismatch of macOS version between the kernel cache (kernel + extensions) and the macOS root FS.
99+
To figure out where the boot process is stuck, you can start a serial utility like minicom/picocom and the like, with 115200 baud rate (something like `picocom -b 115200 /dev/ttyACM1`).
100+
Be patient when you are booting with one cpu and the hypervisor. Depending on what you are tracing, it is slower than normal and it is expected.
71101

72-
Here are some numbers from some experiment with macOS 11.5.2 and m1n1 version commit bd5211909e36944cb376d66c909544ad23c203bc:
73-
* From run_guest command launched(t0) to start loading kernel: 9s
74-
* From t0 to login screen(without keyboard nor mouse cursor moving first): around 2min.
75-
* With keyboard and mouse cursor moving: around 2min35s.
76-
* From password entered to desktop and menu bar: around +2min.
102+
Here are some numbers from some experiment with macOS `11.5.2` and m1n1 version commit `bd5211909e36944cb376d66c909544ad23c203bc`:
103+
- From run_guest command launched (`t0`) to start loading kernel: 9s.
104+
- From `t0` to login screen (without keyboard nor mouse cursor moving first): around 2min.
105+
- With keyboard and mouse cursor moving: around 2min35s.
106+
- From password entered to desktop and menu bar: around +2min.
77107

78108
## Running the stock macOS kernel from a macOS install
79109

80-
1. Boot into macOS
81-
2. Find the `kernelcache`, it is at ```/System/Volumes/Preboot/(UUID)/boot/(long hash)/System/Library/Caches/com.apple.kernelcaches/kernelcache```
82-
3. Make a copy of this file somewhere
83-
4. Get (or build) a copy of img4tool (https://github.com/tihmstar/img4tool)
110+
1. Boot into macOS.
111+
2. Find the `kernelcache`, it is at `/System/Volumes/Preboot/(UUID)/boot/(long hash)/System/Library/Caches/com.apple.kernelcaches/kernelcache`
112+
3. Make a copy of this file somewhere.
113+
4. Get (or build) a copy of img4tool (https://github.com/tihmstar/img4tool).
84114
5. Extract the im4p image:
85-
```img4tool -e -p out.im4p kernelcache```
115+
116+
img4tool -e -p out.im4p kernelcache
117+
86118
6. Extract the machO from the im4p:
87-
```img4tool -e -o kernel.macho out.im4p```
119+
120+
img4tool -e -o kernel.macho out.im4p
121+
88122
7. You can now run macOS in a similar manner as shown above (just no debug DWARF):
89-
```python3 proxyclient/tools/run_guest.py <PATH_TO_EXTRACTED_MACHO> -- "debug=0x14e serial=3 apcie=0xfffffffe -enable-kprintf-spam wdt=-1 clpc=0"```
123+
124+
python3 proxyclient/tools/run_guest.py \
125+
<PATH_TO_EXTRACTED_MACHO> \
126+
-- "debug=0x14e serial=3 apcie=0xfffffffe -enable-kprintf-spam wdt=-1 clpc=0"
90127

91128
## Updating your m1n1 hypervisor tree
92129

93-
The hypervisor/m1n1 ABI is *not* stable. If you have installed a fresh m1n1 build as above, you can use `run_guest.py` directy to save some time. However, as soon as you update your m1n1 git tree, you *must* build the updated m1n1 and run `python tools/chainload.py -r ../build/m1n1.bin` before `run_guest.py`, to make sure the ABI is synced. Failure to do this will lead to random errors/crashes due to ABI mismatches.
130+
The hypervisor/m1n1 ABI is *not* stable. If you have installed a fresh m1n1 build as above, you can use `run_guest.py` directy to save some time. However, as soon as you update your m1n1 git tree, you *must* build the updated m1n1 and run
131+
```
132+
python tools/chainload.py -r ../build/m1n1.bin
133+
```
134+
before `run_guest.py`, to make sure the ABI is synced. Failure to do this will lead to random errors/crashes due to ABI mismatches.
94135

95136
## Using GDB/LLDB
96137

@@ -112,4 +153,4 @@ process connect unix-connect:///tmp/.m1n1-unix
112153
Do not run hypervisor console commands interfering with GDB/LLDB, or they will be out-of-sync. For example, do not edit breakpoints from both of hypervisor console and GDB/LLDB at the same time.
113154

114155
# Sources
115-
Source for the kernelcache creation: https://kernelshaman.blogspot.com/2021/02/building-xnu-for-macos-112-intel-apple.html
156+
Source for the kernelcache creation: [https://kernelshaman.blogspot.com/2021/02/building-xnu-for-macos-112-intel-apple.html](https://kernelshaman.blogspot.com/2021/02/building-xnu-for-macos-112-intel-apple.html)

0 commit comments

Comments
 (0)