Skip to content

python: fix iterators failing on empty lists#1067

Merged
igaw merged 2 commits intolinux-nvme:masterfrom
martin-belanger:python-iterators
Sep 8, 2025
Merged

python: fix iterators failing on empty lists#1067
igaw merged 2 commits intolinux-nvme:masterfrom
martin-belanger:python-iterators

Conversation

@martin-belanger
Copy link
Copy Markdown
Contributor

@martin-belanger martin-belanger commented Sep 4, 2025

This is my attempt to fix the issue reported by @hreinecke: #1064

With this PR, subsystems(), controllers(), etc, are now implemented as native Python iterators (using yield).

After confirming with Hannes I also removed all the iter elements (host_iter, subsystem_iter, etc) that were no longer needed. I kept the removal of the iter elements as a separate commit just in case we need to revert the removal.

The python bindings implement hosts(), subsystems(),
controllers(), and namespaces() as iterators. However,
these iterators all fail on empty lists.

This hopefully fixes this issue.

Signed-off-by: Martin Belanger <[email protected]>
@hreinecke
Copy link
Copy Markdown
Collaborator

That looks indeed far cleaner. Have you checked whether you can remove the iter elements? They should not be needed now.

@martin-belanger
Copy link
Copy Markdown
Contributor Author

Have you checked whether you can remove the iter elements? They should not be needed now.

I will try removing the iter elements and I'll run some tests. I just needed your confirmation that they were not needed for something else.

@hreinecke
Copy link
Copy Markdown
Collaborator

No, they are not. I've removed them on my local branch and everything worked. Would've pushed them but that would have created a new PR I guess, so I didn't.

This is just a bit of clean up. After reworking iterators by using
actual Python code instead of C++, we don't need to keep these
"iter" elements: host_iter, subsystem_iter, ctrl_iter, ns_iter.

Signed-off-by: Martin Belanger <[email protected]>
@martin-belanger martin-belanger changed the title [WIP] python: fix iterators failing on empty lists python: fix iterators failing on empty lists Sep 5, 2025
@martin-belanger
Copy link
Copy Markdown
Contributor Author

@hreinecke @igaw - Done with the changes. Removed the WIP label on this PR.

Copy link
Copy Markdown
Collaborator

@hreinecke hreinecke left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks good to me.

Comment thread libnvme/nvme.i
struct nvme_ns * nvme_subsystem_first_ns(struct nvme_subsystem * s);
struct nvme_ns * nvme_subsystem_next_ns(struct nvme_subsystem * s, struct nvme_ns * n);
struct nvme_ns * nvme_ctrl_first_ns(struct nvme_ctrl * c);
struct nvme_ns * nvme_ctrl_next_ns(struct nvme_ctrl * c, struct nvme_ns * n);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these deceleration necessary? I see we include tree.h right above.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that python-sigsegv-during-gc fails without these deceleration. I think I don't understand how the SWIG compiler works :)

@martin-belanger
Copy link
Copy Markdown
Contributor Author

martin-belanger commented Sep 8, 2025

SWIG is a weird beast...

There are two types of includes.

  1. #include
  2. %include

The first one is to be added to the generated C/C++ code. This is so that the generated C/C++ code will find the definitions and prototypes that it needs to build the C/C++ code.

The second one is to tell SWIG that we want to generate Python wrappers for everything found in the included file. Since we don't want to generate Python wrappers for everything in tree.h, we don't use this type of %include. Instead, we list the exact functions that we want to wrap.

This is the part that I find a bit annoying with SWIG. It would be simpler if we just wrapped everything, but then the generated code would contain a lot of things we don't really need in Python. We could also use a SWIG flag (SWIG) that is used to mark what we want to include/exclude from tree.h itself (i.e. #ifdef SWIG). But that would mean littering tree.h with a bunch of #ifdef SWIG, and I don't think we want to go down that route...

@igaw
Copy link
Copy Markdown
Collaborator

igaw commented Sep 8, 2025

Thanks for explaining. Yeah, SWIG seems tricky to use.

@igaw igaw merged commit 2c90d97 into linux-nvme:master Sep 8, 2025
12 checks passed
@igaw
Copy link
Copy Markdown
Collaborator

igaw commented Sep 8, 2025

Thanks! And I do like to get rid of these C++ iters. First I thought it's odd to add Python code but given we can get rid of this C++, I am for it!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants