(INF-2987) Create script to mass import users into CO Manage Registry from files#29
Open
williamnswanson wants to merge 13 commits into
Open
(INF-2987) Create script to mass import users into CO Manage Registry from files#29williamnswanson wants to merge 13 commits into
williamnswanson wants to merge 13 commits into
Conversation
Also remove old commented out version
…d COmanage schema add EOF newline
There was a problem hiding this comment.
Pull request overview
Adds a new bulk-import script to create COmanage Registry CO Person records from JSON input, including related objects (roles, group memberships, SSH keys, Unix cluster accounts/groups), supported by new schema-building helpers and some new COmanage utility APIs.
Changes:
- Added
mass_person_create_modify.pyto read an input data file + mapping file and create/provision CO Person-related records in COmanage. - Added
comanage_person_schema_utils.pywith helpers for constructing Core API v1-style CO Person payloads. - Extended
comanage_utils.pywith Core API v1 people helpers, new retry/fast-fail HTTP handling, and group creation helper.
Reviewed changes
Copilot reviewed 3 out of 4 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| mass_person_create_modify.py | New mass import script to create CO People, groups, Unix accounts, and provision them. |
| comanage_utils.py | Adds HTTP fast-fail exception type, Core API people helpers, identifier update helper, and CO group creation helper. |
| comanage_person_schema_utils.py | New module defining payload templates and helper constructors for CO Person-related objects. |
| .gitignore | Ignores Python __pycache__ artifacts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+33
to
+34
| # HTTP return codes we shouldn't attempt to retry | ||
| HTTP_NO_RETRY_CODES = {401, 404, 405, 500} |
Comment on lines
+209
to
+210
| return call_api3(PUT, "/api/v2/identifiers" % id_, id_data, endpoint, authstr, ) | ||
| #return call_api3(PUT, "identifiers/%s.json" % id_, id_data, endpoint, authstr) |
| "RequestType" : "CoGroups", | ||
| "Version" : "1.0" | ||
| } | ||
| return call_api3(POST, "co_groups/.json", data, endpoint, authstr) |
Comment on lines
+169
to
+173
| def co_person_org_id( | ||
| osg_co_id, name, organization="", department="", title="", affiliation="member", id_list=[] | ||
| ): | ||
| #org_id = {"co_id" : osg_co_id} | ||
| org_id = ORG_IDENTITY.copy() |
Comment on lines
+66
to
+71
| try: | ||
| user, passwd = utils.getpw(options.user, passfd, passfile) | ||
| options.authstr = utils.mkauthstr(user, passwd) | ||
| except PermissionError: | ||
| usage("PASS required") | ||
|
|
Comment on lines
+168
to
+171
| identifiers_list = co_person_record["Identifier"] | ||
| username = next((item["identifier"] for item in identifiers_list if item["type"] == "osguser")) | ||
| uid = next((item["identifier"] for item in identifiers_list if item["type"] == "uid")) | ||
| description = f"Unix Cluster Group for {username}" |
Comment on lines
+73
to
+87
| def read_data_dump(): | ||
| data_json = [] | ||
| with open(options.input_file, 'r', encoding='utf-8') as input_file: | ||
| data_json = json.load(input_file) | ||
| for entry in range(len(data_json)): | ||
| for key_index in range(len(data_json[entry]["public_keys"])): | ||
| key = data_json[entry]["public_keys"][key_index] | ||
| key_sections = str(key).split() | ||
| if len(key_sections) >= 2: | ||
| data_json[entry]["public_keys"][key_index] = {"type" : key_sections[0], "pkey" : key_sections[1]} | ||
| if len(key_sections) >= 3: | ||
| data_json[entry]["public_keys"][key_index].update({"authenticator" : key_sections[2]}) | ||
| with open(options.mapping_file, 'r', encoding='utf-8') as mapping_file: | ||
| mapping_json = json.load(mapping_file) | ||
| return data_json, mapping_json |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Takes a file with a list of maps of key:value pairs and creates a CO Person record in the COmanage Registry (along with CO Roles, CO Group Memberships, SSH Keys, Unix cluster accounts and groups, ect)
The script doesn't currently have a way to know what parts of the mapping file should be used for what attributes of what type of record, so the
build_co_person_record()function will need to be modified to suit the specific import you're trying to do, what data you have for the users, ect.