Test Update - #14
Open
maxpromer wants to merge 6658 commits into
Open
Conversation
|
The current version (before the incoming merge) has a bug while trying to compile ports/unix. I have to edit the code of the AXTLS library: -------------------------- ssl/os_port_micropython.h -------------------------- #define TTY_FLUSH() -#include "../../../extmod/crypto-algorithms/sha256.h" #define SHA256_CTX CRYAL_SHA256_CTX |
Avoid getting fancy with MICROPY_MALLOC_USES_ALLOCATED_SIZE and make the function calls with/without this define explicit and readable. Signed-off-by: Phil Howard <[email protected]>
Run code formatting to apply rules to: * drivers/**/*.[ch] * examples/**/*.[ch] * extmod/nimble/**/*.[ch] * shared/libc/*.[ch] Signed-off-by: Phil Howard <[email protected]>
Signed-off-by: Damien George <[email protected]>
This commit updates the `make-memzip.py` utility to finish the migration to Python 3.x. The code still wasn't fully aware of how Python unified bytes and strings when it comes to I/O API. These changes help with that. Signed-off-by: Alessandro Gatti <[email protected]>
This commit makes the `make-memzip.py` utility more resilient in certain usage cases, which weren't accounted for in the original source. In some Linux distributions the `zip` command may not be available, although `unzip` could be. Unless the `zip` package is installed this script will fail with a generic "file not found" error, which has been addressed in these changes set. Also, the `zip` command by default adds a `.zip` suffix to archives, which wasn't accounted for. So, running `./make-memzip -z z1 -c z2 .` would fail, as the script would build `z1.zip` whilst the header generator attempted to open `z1` instead. This has now been fixed. Signed-off-by: Alessandro Gatti <[email protected]>
This commit modifies the native modules build system to use different
directories for different architectures.
Previously, the natmod build system would unconditionally put all build
data into a single directory (usually "$(PWD)/build" unless overridden),
and the static library runtime cache into another directory
("$(PWD)/.mpy_ld_cache"). That works if building is always done for a
single architecture or if a full clean is performed before switching
architectures, which sometimes is not wanted (collecting the runtime
cache may take quite some time depending on the module complexity, for
example).
With these changes, both the build directory and the runtime cache
directories are marked with the name of the architecture they target, so
for example "$(PWD)/build" becomes "$(PWD)/build_x64" (the same happens
to the cache directory name).
Signed-off-by: Alessandro Gatti <[email protected]>
This commit makes a minor change to the native module build scripts, to make the intermediate MPY file keep the same file name it will have once the final pre-processing step takes place. A MPY file will retain its file name as the first QSTR entry in the file, and before these changes the name being written would be the one used for the intermediate MPY output file (ie. "$BUILDDIR/$MODNAME.native.mpy"). The `mpy-tool.py` preprocessing pass would keep that string intact but rename the file, so things wouldn't match in error tracebacks. Signed-off-by: Alessandro Gatti <[email protected]>
This commit adds an option to the MPY linker to brand generated file with a custom internal file name rather than the raw output path provided in the command line. MPY files, as part of their preambles, will contain a path name as the first entry in the constant string pool. However that may contain unneeded data (ie. sometimes the plain file name is enough) or it may have sensitive data in there (if an alternate build path is provided it may contain product code names or client names which should probably not be disclosed). Until now there was no provision to let users use a different internal path name. A new option was added to `mpy_ld.py`: "--source-name" which, if passed, will write the given string in the module file. If said option is not provided, generated MPY files will still carry the raw command line output path but stripped of all leading path segments. The example natmod makefiles have been updated to always provide that option whenever the final file name on the filesystem wouldn't match the internal MPY file name. Signed-off-by: Alessandro Gatti <[email protected]>
MICROPY_PY_LWIP_PPP is used in only one place and is generally set to
the value of MICROPY_PY_NETWORK_PPP_LWIP. Remove the former.
extmod/lwip-include/lwipopts_common.h: Replace MICROPY_PY_LWIP_PPP
with MICROPY_PY_NETWORK_PPP_LWIP.
ports/mimxrt/mpconfigport.h: Remove redundant MICROPY_PY_LWIP_PPP.
ports/rp2/mpconfigport.h: Remove redundant MICROPY_PY_LWIP_PPP,
default MICROPY_PY_NETWORK_PPP_LWIP to
MICROPY_PY_LWIP.
ports/stm32/mpconfigport.h: Remove redundant MICROPY_PY_LWIP_PPP.
Signed-off-by: Phil Howard <[email protected]>
mpconfigport.h: Default MICROPY_PY_NETWORK_PPP_LWIP to 1. Signed-off-by: Phil Howard <[email protected]>
This updates the PATHS to include: - everything under `extmod/` - everything under `py/` (in case one day there are subdirs there...) - everything under `shared/` This makes sure that any C source files added anywhere under these directories will be formatted. Signed-off-by: Damien George <[email protected]>
These need to be formatted due to the parent commit. Signed-off-by: Damien George <[email protected]>
This reverts commit 6552836. Signed-off-by: Angus Gratton <[email protected]>
This seems like it's only really a problem on Debug builds, but I think can't hurt to increase it on all windows builds. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>
This removes the last usage of mp_binary_set_val_array_from_int(). It will be a little slower, but shouldn't be measurably so compared to the ADC sampling. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>
If >1 unittest-enabled module is included, the results of the merged module won't match (as it runs some previously registered tests again). Signed-off-by: Angus Gratton <[email protected]>
This PR was originally a cherry-pick of CircuitPython commits 095c844, d103ac1, c592bd6 and 8664a65. However, substantial additional changes so the implementation has diverged a lot from CircuitPython's: - Keep CPython >= 3.11 defaults for int.to_bytes(), consistent with 80c5e76 and 0b432b3. - Refactors to reduce the code size impact of this change. Various code paths are now funneled into new function mp_obj_int_to_bytes() and existing function mp_binary_set_int(), except where a simple assignment is used for performance reasons (i.e. array, moductypes). - Keep the MicroPython behaviour of not overflow checking assignments to arrays, bytearrays, etc. - but enable overflow checks as part of MicroPython V2.0. - Update tests to work with the new behaviour (similar to CPython) and add coverage for new code and some corner cases. Some tests are converted to unittest so we can easily verify both current and V2.0 overflow. This work was funded through GitHub Sponsors. Signed-off-by: Angus Gratton <[email protected]>
Generated `qstr.i.last` files (output from the "pp" phase) can be very large due to inclusion of large HAL headers. Yet only a small amount of this data is actually needed by the "split" phase of `makeqstrdefs.py`. This commit improves the situation by filtering out unneeded lines as early as possible: in the "pp" phase the C preprocessor runs, its output is parsed via a pipe connection, and only those lines that are needed are written out to the `qstr.i.last` file. When the "split" phase runs it then sees only the necessary lines (it does further regex filtering to get exactly the lines it needs for the given mode). Windows already does this filtering (it does not use `makeqstrdefs.py` for the preprocessor phase) in its `ConcatPreProcFiles` function, and that was added long ago in 29c8c8a. This change significantly reduces the output build size and improves the build speed: - stm32 BOARD=PYBV10: build output goes from 152M down to 32M, and build is 10% faster (29s down to 26s). Building all stm32 boards, the total build size goes from about 11GB down to about 2.5GB. - rp2 BOARD=RPI_PICO: build output goes from 87M down to 39M, and build is about 5% faster (23s down to 21.6s). - esp32 BOARD=ESP32_GENERIC: build output goes from 307M down to 230M, and build is about 6% faster (101s down to 95s). Tested all three ports above, that the files in build/genhdr are equivalent (except of course `qstr.i.last`), and that PYBV10 is binary equivalent before and after this change. Signed-off-by: Damien George <[email protected]>
This adds a new `--trace-output` option to `run-tests.py` which mimics the same option already in `run-multitests.py`. Using it, the output from the test running on a board is printed to stdout as it is received. This allows easier debugging of tests, to see the output in real time. For tests that have complicated run parameters, eg using via-mpy or needing a target wiring script, running them standalone (eg with `mpremote`) is not possible, so using `run-tests.py` becomes mandatory, hence the need to easily see the output without having to view the result file separately at the end. It's also useful to watch tests that pass but are long running. Signed-off-by: Damien George <[email protected]>
Similar to other classes in the `machine` module, `SDCard` is now there for a port to use if it enables MICROPY_PY_MACHINE_SDCARD. Signed-off-by: Damien George <[email protected]>
This is now provided by the common one in `extmod/modmachine.c`. Signed-off-by: Damien George <[email protected]>
To unify this with other ports, and to enable the common `machine.SDCard` entry in `extmod/modmachine.c`. Signed-off-by: Damien George <[email protected]>
To unify this with other ports, and to enable the common `machine.SDCard` entry in `extmod/modmachine.c`. Signed-off-by: Damien George <[email protected]>
This aligns the stm32 port with other ports that have `machine.SDCard`. The same `SDCard` class is used for this and the existing `pyb.SDCard`, and its API matches the standard one. Signed-off-by: Damien George <[email protected]>
Add a minimal `network.WLAN` CSI API for ESP32 builds with CSI enabled. Also enable ESP-IDF CSI support in the standard generic `ESP32`, `ESP32-C3`, `ESP32-S3`, `ESP32-C5`, and `ESP32-C6` board definitions. Signed-off-by: Francesco Pace <[email protected]>
Add a board definition for the Seeed Studio XIAO ESP32C3 (ESP32-C3 with 4MB flash). The REPL is on the native USB Serial/JTAG interface, leaving UART0 free for the application. Signed-off-by: Pavel Revak <[email protected]>
Add a board definition for the Seeed Studio XIAO ESP32C5 (ESP32-C5 with 8MB flash and 8MB quad SPIRAM). The REPL is on the native USB Serial/JTAG interface, leaving UART0 free for the application. Signed-off-by: Pavel Revak <[email protected]>
Add a board definition for the Seeed Studio XIAO ESP32S3 (ESP32-S3 with 8MB flash and 8MB octal SPIRAM). The REPL is on the native USB interface, leaving UART0 free for the application. The same firmware also runs on the XIAO ESP32S3 Plus: the flash size is auto-detected, and the Plus's extra pins (D11-D19) are included in pins.csv, which does not affect the standard board. Signed-off-by: Pavel Revak <[email protected]>
By default the new I2C driver is enabled for esp-idf version >= 5.5.2. With v5.5.2 and up the the additional bus probing is not needed any more. It was needed because transfers to a non-existing address caused an error message to be printed instead of just returning an error code. With esp-idf v5.5.2 it returns just an error code. For 0-length writes as being used by i2c.scan(), an additional code path using i2c_master_execute_defined_operations() is needed to avoid an error message. Tested with a Generic ESP32 and esp-idf v5.5.1, v5.5.2 and v5.5.4. Tested as well using the new driver with v5.5.1. Tested V5.5.2 with Generic ESP32, ESP32S2, ESP32S3, ESP32C2, ESP32C3, ESP32C5, ESP32C6, ESP32P4. Driving an external device as I2C controller. There is a slight inconsistency: reading 0 bytes from a non-existing address return an empty bytes object and does not raise an error. Writing 0 bytes to a non-existing address returns an error. That case is used by i2c.scan(). Signed-off-by: robert-hh <[email protected]>
This flash size of this board was corrected from 4MiB down to 2MiB in pico-sdk 2.3.0, in commit raspberrypi/pico-sdk@0da73ec So, reduce the filesystem size in order that the firmware fits. The size of 1408k was chosen to match SEEED_XIAO_RP2040. Signed-off-by: Damien George <[email protected]>
This board is tight on flash and sets the feature level to "core". That excludes `memoryview` which is needed for the new `machine.mem_backup()` feature. Both `memoryview` and `mem_backup` are useful features, so enable the former to get them both working. Signed-off-by: Damien George <[email protected]>
closes: #18657 Signed-off-by: Jos Verlinde <[email protected]>
fixes: #18658 Signed-off-by: Jos Verlinde <[email protected]>
Adds a Modern Windows console implementation that can write raw UTF-8 bytes directly (like POSIX) Improve the Legacy consol to use an incremental decoder to handle split UTF-8 sequences within writes. fixes #15228 Signed-off-by: Jos Verlinde <[email protected]>
RemoteCommand.wr_bytes(): Now correctly encodes strings to UTF-8 bytes before calculating length. Based on the suggestion by @felixdoer Fixes issue #13055. Signed-off-by: Jos Verlinde <[email protected]>
Signed-off-by: Jos Verlinde <[email protected]>
Reduces code duplication while allowing different configurations. Signed-off-by: Jos Verlinde <[email protected]>
Add regressions tests for utf-8 and unicode. Signed-off-by: Jos Verlinde <[email protected]>
When soft_timer_heap is NULL the expression &soft_timer_heap->pairheap is undefined behaviour in both C and C++. This is harmless today only because pairheap is at offset 0, so the expression folds to NULL, but a struct reorder would corrupt the heap. The same pattern occurs in soft_timer_insert(), remove(), handler(), deinit() and gc_mark_all(). This patch fixes the UB by introducing a safe pairheap accessor that avoids the undefined behaviour of &e->pairheap when e is NULL, while guaranteeing that pairheap is always the first member, via static assert. Signed-off-by: iabdalkader <[email protected]>
Make the function machine_bootloader_rtc() available to other modules by declaring it in mpconfigport.h, when selected via its #define. This is needed for the Arduino Nano ESP32 board to be able to call this function from the board_init.c file. Signed-off-by: Luca Burelli <[email protected]>
The custom machine.bootloader() function in the Nano ESP32 board currently calls the Arduino bootloader to allow DFU updates on the board. However there are situations where the hardware bootloader may still be useful, for example when a full flash is required. Make it so calling `machine.bootloader(1)` will fallback to the default bootloader; anything else will result in the one by Arduino being started. Signed-off-by: Luca Burelli <[email protected]>
So it can be skipped if the target has unicode enabled but doesn't support f-strings. For example ADAFRUIT_ITSYBITSY_M0_EXPRESS. Signed-off-by: Damien George <[email protected]>
Tests are run from within their directory, so extmod is actually two levels up, not one. Use `base_path()` to get the correct value, following how it's done in `run-tests.py`. Signed-off-by: Damien George <[email protected]>
Instead of "latin" which is unsupported, and now an error since commit 770bb3b Signed-off-by: Damien George <[email protected]>
Newer versions of `esptool` (eg v5.2.0) require the `--chip` option to be specified. And older versions (eg v4.5) accept it, so this change is backwards compatible. Signed-off-by: Damien George <[email protected]>
Add a new c_module() function to the manifest API that lets a manifest declare which user C module directories should be included in the build. This pairs with the build system changes that pick up the resulting paths and feed them into USER_C_MODULES. The function validates the path exists, is a directory, and contains a micropython.mk or micropython.cmake, and supports the usual \$(MPY_DIR), \$(BOARD_DIR), \$(PORT_DIR), \$(MPY_LIB_DIR) substitutions. Unresolved \$(VAR) tokens raise a clear error rather than failing later. makemanifest.py gains --list-c-modules which prints one path per line so the build system can pick them up (newline output handles paths that contain whitespace on the cmake side). Signed-off-by: Andrew Leech <[email protected]>
When MICROPY_FROZEN_MANIFEST is set, the build system runs makemanifest.py with --list-c-modules to extract C module paths from the manifest and appends them to USER_C_MODULES so they go through the same code path as user modules specified on the command line. The Make side lives in py/manifest.mk (included from py/py.mk) and the CMake side lives in py/manifest.cmake (included from py/usermod.cmake). Both share defaulting for the MICROPY_MANIFEST_* substitution variables so a manifest can reference $(MPY_DIR) etc. consistently on either side. The make-side extraction is gated on the build target needing it and on micropython-lib being initialised, and surfaces makemanifest.py errors via .SHELLSTATUS. The cmake side uses FATAL_ERROR to match. Signed-off-by: Andrew Leech <[email protected]>
Move the MICROPY_USER_FROZEN_MANIFEST resolution to run right after the board config (rather than near the end of CMakeLists.txt) so it's applied before py/usermod.cmake is included; py/manifest.cmake (invoked from usermod.cmake) needs the final MICROPY_FROZEN_MANIFEST value to extract c_module() paths. Also quote FROZEN_MANIFEST when forwarding it from the Makefile to cmake so paths containing whitespace round-trip correctly. Signed-off-by: Andrew Leech <[email protected]>
Quote FROZEN_MANIFEST when passing it through IDFPY_FLAGS so paths containing whitespace round-trip correctly to the cmake configure step; py/manifest.cmake (auto-included via py/usermod.cmake) needs the resolved value to extract c_module() paths from the manifest. Signed-off-by: Andrew Leech <[email protected]>
The micropython.cmake referenced examplemodule.c but the source file is named modexamplepackage.c. The subpackage example wasn't built via cmake before, so the typo went unnoticed. Signed-off-by: Andrew Leech <[email protected]>
Document the new c_module() function in the manifest reference, including the FROZEN_MANIFEST prerequisite, additive interaction with USER_C_MODULES, and the make-side whitespace limitation. Add a corresponding section to the cmodules developer guide so the feature is discoverable from both entry points. Signed-off-by: Andrew Leech <[email protected]>
Provides tests/tools/manifest_c_module.py, a shared frozen manifest that pulls in the example user C modules via c_module() and includes the port's default board manifest via $(PORT_DIR). Wires it up in ci.sh so rp2 (RPI_PICO2), stm32 (PYBV11) and esp32 (SPIRAM variant) each exercise the c_module() manifest path alongside the legacy USER_C_MODULES= command-line variable. Signed-off-by: Andrew Leech <[email protected]>
The unix coverage variant set USER_C_MODULES from mpconfigvariant.mk to build the example user C modules into the test binary. Move that setting into the coverage manifest as a c_module() call to exercise the new manifest API end-to-end as part of the standard coverage test build. Signed-off-by: Andrew Leech <[email protected]>
`NVIC_SetPriority` is an inline function with separate paths for IRQ number negative vs non-negative. When called here to set the GPIO IRQ priority, gcc sees that the IRQ number could potentially be NotAvail_IRQn=-128 (although in practice that never occurs). Gcc then issues an array-out-of-bounds error due to the negative code path only allowing down to -16 for the IRQ number. Fix that by forcing the IRQ number to be positive, similar to how stm32 does it. That also allows the compiler to optimise away the non-negative code path. Signed-off-by: Damien George <[email protected]>
These variables are never used by the function. Signed-off-by: Damien George <[email protected]>
MP_STREAM_POLL was previously just returning MP_STREAM_ERROR. Signed-off-by: Damien George <[email protected]>
Signed-off-by: Damien George <[email protected]>
The F4 and F7 HAL have a volatile debug variable that is only ever set. Signed-off-by: Damien George <[email protected]>
This reverts commit 4edef87. The `mod_network_init()` call was already moved before `boot.py` in commit 7d77d3e. Signed-off-by: Damien George <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.