NGINX Unit by gsfr · Pull Request #992 · scitran/core

gsfr · 2017-11-14T00:32:05Z

This is work in progress. Help wanted.

Scope proposal:

Ability to run core's own test suite on NGINX Unit and Alpine Linux.
Ability to trivially bring up a fully functional core, including mongo, to be used for testing of other components, such as reaper or cli.

Comments and/or requirement definitions welcome.

TO DO

Revert Have Travis push to Docker Hub for this branch

gsfr · 2017-11-14T00:34:36Z

@ambrussimon I would like to use the setup.py/requirements.txt pattern that you have established in reaper, but I don't know how to handle the necessary dependency_links. Any ideas?

kofalt

Looks interesting!

Do you have any benchmarks or observed behavior?

kofalt · 2017-11-14T16:35:33Z

-uwsgi==2.0.13.1
-webapp2==2.5.2
-WebOb==1.5.1
-git+https://github.com/flywheel-io/[email protected]#egg=gears


I use this file for CI, so I would need an equivalent command. I'll want to try things out with this branch if it gets closer to merging.

We will make sure that you don't need any of that anymore. You should able to simply stand up a core and use it. No internal knowledge required.

Sounds good, as long as the container checks itself on boot. Frequently I want to CI against a branch whose container (incl updated libraries) has not built yet.

That's a good point. We'll need to have images available for arbitrary branches of core to facilitate testing of interacting repos, such as reaper and cli. We typically build images for master and all tags. I think that's enough (rather than building for all branches) and keeps clutter at bay. Would that work for you, @kofalt?

Discussed this out of band - basically, as long as any CI container can update itself on boot to the tip of a branch (incl updated libraries) we're fine.

kofalt · 2017-11-14T16:36:04Z

+
+RUN curl -L https://github.com/nginx/unit/archive/master.tar.gz | tar xz --strip-components 1
+RUN ./configure --prefix=/usr/local --modules=lib --state=/var/local/unit --pid=/var/unit.pid --log=/var/log/unit.log \
+ && ./configure python \


We should check the configure options, nginx-web (or whatever) has quite a few good ones.

kofalt · 2017-11-14T16:37:13Z

+
+dependency_links = [
+    'git+https://github.com/flywheel-io/[email protected]#egg=gears',
+]


I take it this entry can't be in the above array?

Not that I could tell, but @ambrussimon will look further into it.

gsfr · 2017-11-14T18:41:48Z

@kofalt No benchmarks yet. Still at the 'getting it running' stage. We'll let you know when this is ready to be tested against.

ryansanford · 2017-11-16T18:59:46Z

@gsfr Isn't the definition of this PR being ready that it's not a "Breaking Change" ?

gsfr · 2017-11-17T03:01:28Z

@ryansanford Won't be a breaking chance from the user point of view, but will be for parts of our infra. Removing tag.

nagem · 2017-11-29T17:28:05Z

Remove abao

🎉🎉🎉🎉🎉🎉🎉🎉🎉

gsfr · 2017-12-14T22:17:32Z

 elasticsearch==5.3.0
 enum34==1.1.6
+git+https://github.com/flywheel-io/[email protected]#egg=gears
+ipython


I'm guessing you want IPython for debugging, but it's such a quick install and I'd rather not bake it into the image.

Just seeing where IPython came from. I'm taking it out.

gsfr · 2017-12-14T22:17:59Z

+ipython
 jsonschema==2.6.0
-Markdown==2.6.5
+markdown==2.6.5


The reason for the capitalization is to match the output of pip freeze, so that it can be copied verbatim.

gsfr · 2018-01-02T21:59:50Z

@ambrussimon nginx/unit#65 has been closed. We should now be able to drop your coverage endpoint workaround. However, it appears that Unit's introduction of HTTP Keep-Alive has broken our tests. The previous commit (nginx/unit@497faf1) is the last one that seems to work.

ambrussimon · 2018-01-09T16:40:22Z

@gsfr Just rebased (fixed conflicts), no functional change.

(needs to be undone before merge)

kofalt · 2018-01-31T20:02:08Z

Ref flywheel-io/sdk#80 for current endpoint status!

gsfr · 2018-02-01T02:29:57Z

Thanks, @kofalt.

@ambrussimon Please take a look at the failing SDK tests. Getting those to pass would be a great milestone for this PR.

kofalt · 2018-02-01T15:33:39Z

If Ambrus or anyone else has a look at playing with those, note the docs to help you out:
https://github.com/flywheel-io/sdk#testing

In particular, the -run flag and SdkTestDebug env variable may be of interest.

gsfr · 2018-03-10T01:14:26Z

@ambrussimon In trying to get flywheel-io/sdk#80 going, I added upgrade_schema to dev+mongo.sh. Unfortunately, that seems to break some tests. Is it ok/desirable to have this? Can we fix the tests?

gsfr added Breaking Change Help Wanted Tech Debt labels Nov 14, 2017

gsfr assigned ryansanford and ambrussimon Nov 14, 2017

kofalt reviewed Nov 14, 2017

View reviewed changes

gsfr removed the Breaking Change label Nov 17, 2017

ambrussimon force-pushed the nginx-unit branch from 0c74af1 to 9ccd114 Compare November 17, 2017 16:55

gsfr force-pushed the nginx-unit branch 2 times, most recently from 7743faf to def94cf Compare November 22, 2017 23:12

ambrussimon force-pushed the nginx-unit branch from 5efc4b0 to 5ebe6d8 Compare November 29, 2017 13:45

ambrussimon force-pushed the nginx-unit branch from 59350a0 to 12fd233 Compare December 14, 2017 13:51

gsfr commented Dec 14, 2017

View reviewed changes

gsfr force-pushed the nginx-unit branch 2 times, most recently from e03df1f to b2cb26e Compare December 20, 2017 21:59

ambrussimon force-pushed the nginx-unit branch from b2cb26e to 1303bbd Compare January 9, 2018 16:35

gsfr force-pushed the nginx-unit branch from b163af1 to 454a0fb Compare January 11, 2018 23:43

gsfr and others added 6 commits January 22, 2018 15:30

Crude beginnings

5e86b32

Make test scripts more POSIX-friendly

a3d1637

Requirements overhaul

1989c67

WIP - test script reorg

5da4994

Implement lessons learned in reaper infra

9949a27

Fix URL for integration tests

1ba996b

Purging py-openssl (core will be non-SSL)

6ac0f7e

ambrussimon force-pushed the nginx-unit branch from 005b1f9 to ba51d52 Compare January 22, 2018 14:52

ryansanford self-requested a review January 23, 2018 16:39

gsfr force-pushed the nginx-unit branch from 5118f91 to 96a7fd9 Compare January 30, 2018 19:52

gsfr and others added 12 commits January 30, 2018 12:23

Have Travis push to Docker Hub for this branch

e870763

(needs to be undone before merge)

Run app as unprivileged user

4feab76

Substantial cleanup

35a0678

Pin to nginx/unit/tag/0.4

9f4f367

Force remove of .coverage and htmlcov

a477bcc

Fix runtime coverage save (use /tmp for write permissions)

0379847

Add pre-test cleanup - fix pyc problems for good

a5bb1d8

Simplify testing and update test docs

8558808

Coverage save atexit trial

4222879

Cleanup

384c727

Bump base image version

4348175

Only chown if needed

bb73a8e

gsfr force-pushed the nginx-unit branch from 96a7fd9 to bb73a8e Compare January 30, 2018 20:31

gsfr added 2 commits January 31, 2018 15:45

Clean up bin

28ca260

Rename database.py

3462a9d

gsfr force-pushed the nginx-unit branch from fd4007b to 3462a9d Compare January 31, 2018 21:46

Remove entrypoint

8ac2791

gsfr added 2 commits March 9, 2018 15:36

Add nginx for dev

4030d21

Add upgrade_schema to dev

fe5dbfc

gsfr removed the request for review from ryansanford May 11, 2018 14:58

gsfr unassigned ryansanford and ambrussimon May 11, 2018

Conversation

gsfr commented Nov 14, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gsfr commented Nov 14, 2017

Uh oh!

kofalt left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gsfr commented Nov 14, 2017

Uh oh!

ryansanford commented Nov 16, 2017

Uh oh!

gsfr commented Nov 17, 2017

Uh oh!

nagem commented Nov 29, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gsfr commented Jan 2, 2018

Uh oh!

ambrussimon commented Jan 9, 2018

Uh oh!

kofalt commented Jan 31, 2018

Uh oh!

gsfr commented Feb 1, 2018

Uh oh!

kofalt commented Feb 1, 2018

Uh oh!

gsfr commented Mar 10, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gsfr commented Nov 14, 2017 •

edited

Loading