improve load times w/ fixed gemset iteration

[AprilArcus]

This PR merges @nrser's work to improve load times, as discussed at #84 (comment), with some trivial fixes to an iteration error from my own fork.

[jf]

@AprilArcus @nrser can we leave out 5a1b480 ? Leave the versioning / scheme to me. I'll bump the version when this is done (and I think 3 numbers is enough already).

[jf]

re 5a382ef:

New version adds ~15ms to `ruby -e 'puts "yo"' time :)

sorry, "adds"?

Thanks for adding stuff like rbenv_gemset_debug, btw. For trying to understand what the code does - and I think even for the purpose of code review - it is useful. I will eventually have to remove this in the end before it becomes production.

re the use of rbenv_gemset_lib_loaded: I agree with this approach. I took the same approach for something I wanted to apply to rbenv (you can see rbenv/rbenv#507)

[jf]

@nrser can you explain your comment in 43ca1c1 ? What are "lock executables"? and what is "QB"?

[nrser]

@AprilArcus @nrser can we leave out 5a1b480 ? Leave the versioning / scheme to me. I'll bump the version when this is done (and I think 3 numbers is enough already).

Yeah for sure, I just did that so I had a different version number for my fork and could tell the difference from a quick rbenv gemset version (think I had both the master and my fork installed alongside when I was doing the initial work).

The commits you're looking at are the ones I made as I was writing it. I thought I would come back and rebase it up nice afterwards, but once I realized I wasn't going to have time for that any time soon I just shared my branch, figuring that given the magnitude of performance pain & gain people might want to use it regardless of the mess and lack of test.

[nrser]

re 5a382ef:

New version adds ~15ms to `ruby -e 'puts "yo"' time :)

sorry, "adds"?

Oh, that meant that after the changes using Ruby through rbenv/gemsets only added an extra ~15ms spin-up time (compared to just running the system Ruby without rbenv).

Before the changes, the rbenv/gemsets combo was adding around ~700ms over the system installation (testing with simple ruby -e ... commands).

Eventually I think I ended up spending around 40ms for spin-up, having addressed more of the paths and features than I had at the point of 5a382ef

[nrser]

@nrser can you explain your comment in 43ca1c1 ? What are "lock executables"? and what is "QB"?

Like I said previously, I was just committing as I was working along, thinking it would get rebased for publication. That stuff is just notes to self regarding what was broken on my machine.

The "locks" are this:

https://github.com/nrser/rbenv-lock

lil' rbenv plugin I wrote to "lock" certain Gem executables to specific versions of Ruby.

Like, if you use lunchy or travis or whatever CLI apps and you install them as gems, then you either need to install them in every Ruby version you have in rbenv - in which case they might not even support or work across all of them, and even if they do it makes for a horrible setup and maintenance situation - or they will break when you're using a Ruby or gemset that doesn't have them, either through being in a project directory with a .ruby-version, rbenv shell ... set, etc.

I use Ruby for a lot of personal tools and use a few tools distributed through gems and this annoyed the shit out of me, so I wrote a little plugin to try to fix it.

What the comment's talking about is that QB - a tool I wrote to run Ansible roles against localhost from the CLI - is written in Ruby, and uses rbenv-lock to lock on the the Ruby version it's installed for.

The tricky part is that QB may run tasks that invoke executables that are also written in Ruby, and may even use rbenv-lock, so you can get ENV vars set by rbenv/-gemset/-lock trickling down through descendant processes and screwing up Ruby processes that need a different environment to work.

So exporting RBENV_GEMSET_EXEC_ALREADY was causing problems running child Ruby processes, which totally makes sense.

[nrser]

Thanks for adding stuff like rbenv_gemset_debug, btw. For trying to understand what the code does - and I think even for the purpose of code review - it is useful. I will eventually have to remove this in the end before it becomes production.

rbenv_gemset_debug() only writes output if the $RBENV_GEMSET_DEBUG ENV var is set, so you shouldn't see any debug output unless you turn that guy on, and it's function call, so it's cheap. Do you want to remove it for cosmetic reasons (just don't like seeing/having the calls in the source)?

I don't really do much Bash (and I'm sure it shows), so I'm not sure how projects usually handle logging... but I know at least some tools have CLI options or ENV vars you can use to turn on debug logging in production releases, and I've had it be useful more than once when diagnosing issues.

I see that rbenv just uses their $RBENV_DEBUG flag to set -x, but, man, I've hated wading through the set -x output for anything more than a few-dozen lines of script. Which is why I wrote those logging functions. If that's how it's done though I can get with it.

[AprilArcus]

any chance of merging this this year?

[jf]

Thanks for following up, @AprilArcus . This year? I should think so. I need to find time to review this again.

I thought I stated somewhere the version bump implementation should be up to me... but I don't see it here. If I approve this, I will want to do it without the forced version bump.

[jf]

hi everybody, my sincere, sincere apologies for the long time that this has taken. Ideally, I should NOT have taken this long to get back to you, and for that I am deeply, deeply sorry. I have no excuse. Life hasn't been great for me; but I could have worked on this despite all of it.

Some specific notes:

• I've mentioned not wanting the version bump that's part of this PR. I'm not sure if this was originally there... or if I missed it, but I now see that "Maintainers are allowed to edit this pull request"? So I guess that resolves it. I should be able to do something with that permission.
  • I must confess that I havent really tried this before though (editing a PR); I usually try to work with folks to edit their own PRs; instead of me simply coming in to run over their code. That's my preferred approach.
    • All this to say that this is new territory for me; please be patient! (I'm also saying this, btw, because I am not expecting or assuming that any of you will want to spare the additional time or effort to come back to this after so long).
    • based on a quick search, it would appear that the "maintainer edit" experience wont be a pleasant one (I see, for example, https://github.com/orgs/community/discussions/153403 and https://github.com/orgs/community/discussions/168077)
• fixing this will probably take some time: there's the code to review (and clean up and get up-to-date)... and it's also been a while since I've last looked at the codebase. So give me some time. Thank you!

[jf]

FYI I just finished dealing with an issue in my local setup (I've been bitten by a issue with bash that turned out to be caused by MacPorts that I've been puzzling over for a while) to get rbenv + rbenv-gemset running again.

Just 2 points for now:

ONE, for the record, there can be quite the bit of history behind some of the lines in the scripts, and I need to dig back into the archives to try to recall why something is a certain way.

An example would be the call to $(rbenv which gem), called out at #84 (comment), and in https://github.com/AprilArcus/rbenv-gemset/blob/5f303f6bea1db39be9d23fa9563542cc7ac2d1b1/lib/rbenv-gemset.bash#L238-L256). This goes all the way back to 2013 (!), and at first started with a fix for Rubinius... which then became a more generic fix in the next commit (looking at his name, it would appear that the @felix125 referred to in the commit is now @funal125). I'm not sure why prior to that moment nobody had complained about missing gems for regular Rubies... (perhaps because there weren't too many gems that came with standard Rubies at that time?) but it's definitely something that needs to be added (the latest version of Ruby, 4.0.5, comes with 84 gems per my install).

TWO, at this point in time, I think what is likely to happen with this PR is that I will cherry-pick certain things to put into the master branch... instead of trying to clean up and fix whatever's here (as an example, the shebangs introduced from AprilArcus@1835c77 are actually unnecessary and dont even have any effect).

So for now, I will leave this PR open until I've incorporated everything that I need. There will definitely be attribution though, in case anybody's wondering; not going to make this improvement without giving you guys the credit! Thank you for the patience.

[jf]

@nrser if rbenv-lock is to be supported, I'll need help re-creating a setup where the export of RBENV_GEMSET_ALREADY=yes creates issues. Are you able to help with that? You can refer to #108 for an example.