feat: 新增OH社区项目爬取和鸿蒙系统大版本获取#889
Conversation
- gitcodeOrg: crawl all repos under the GitCode openharmony org into gitcode_projects_t - openharmonyVersion: parse repo tags into adapted HarmonyOS major versions (>= v6.0 release only, beta excluded), stored as JSON array on gitcode_projects_t.openharmony_version (e.g. ["v6.1-release","v6.0-release"]) - register both as sync routes and weekly scheduled jobs - pnpm-workspace: allow build scripts for local (non-docker) startup
Co-authored-by: Cursor <[email protected]>
Jiannan-dev
left a comment
Jiannan-dev
left a comment
There was a problem hiding this comment.
发现 2 个需要修改的问题，已附在对应代码行。
| export async function syncOpenHarmonyCompatibility(options = {}) { | ||
| const { repoFullName, limit, includePreRelease = false } = options; | ||
|
|
||
| const where = { platformType: platformTypes.GITCODE }; |
There was a problem hiding this comment.
这里没有限制到 OpenHarmony 仓库。定时任务在下面用空参数调用 syncOpenHarmonyCompatibility({}) 时，会把 gitcode_projects_t 里所有 GitCode 仓库都拿来请求 tags，并在没有匹配 tag 时把它们的 openharmonyVersion 清成 null。请在未传 repoFullName 时加上 OpenHarmony 专用过滤（例如 ownerName/fullName 前缀或 recordDesc），只处理本次组织同步的仓库。
| `ai_description` JSON NULL, | ||
| `latest_release_tag_name` VARCHAR(255) NULL, | ||
| `latest_release_published_at` VARCHAR(512) NULL, | ||
| `openharmony_version` JSON NULL, |
There was a problem hiding this comment.
这里只改 init.sql 只影响新建库；已有环境的 gitcode_projects_t 不会自动新增 openharmony_version。新接口/定时任务一执行 bulkCreate 或 update 就会因为缺列报 Unknown column 'openharmony_version'。请补一份升级 SQL（例如新的 sql/202506.sql 中 ALTER TABLE gitcode_projects_t ADD COLUMN openharmony_version JSON NULL），并保持初始化脚本同步。
Co-authored-by: Cursor <[email protected]>
Jiannan-dev
left a comment
Jiannan-dev
left a comment
There was a problem hiding this comment.
还有 1 个需要修改的问题，已附在对应代码行。
| on gitcode_projects_t (latest_release_published_at); | ||
|
|
||
| alter table gitcode_projects_t | ||
| add column if not exists openharmony_version json null comment 'OpenHarmony 适配的鸿蒙大版本列表（仅 >= v6.0 的 release），JSON 数组，如 ["v6.1-release","v6.0-release"]' after latest_release_published_at; |
There was a problem hiding this comment.
这里使用了 ADD COLUMN IF NOT EXISTS，但项目的本地数据库镜像是 MySQL 8.0（docker-compose.yml），MySQL 8.0 的 ALTER TABLE 语法只有 ADD [COLUMN] col_name column_definition，不支持这个 MariaDB 风格写法；执行这份迁移会直接语法错误，导致 openharmony_version 仍然不会被加上。请改成 MySQL 兼容写法：要么普通 ADD COLUMN openharmony_version ...，要么用 INFORMATION_SCHEMA.COLUMNS + 动态 SQL 做条件添加。
Co-authored-by: Cursor <[email protected]>
Jiannan-dev
left a comment
Jiannan-dev
left a comment
There was a problem hiding this comment.
还有 1 个需要修改的问题，已附在对应代码行。
| on gitcode_projects_t (latest_release_published_at); | ||
|
|
||
| alter table gitcode_projects_t | ||
| add openharmony_version json null comment 'OpenHarmony 适配的鸿蒙大版本列表（仅 >= v6.0 的 release），JSON 数组，如 ["v6.1-release","v6.0-release"]' after latest_release_published_at; |
There was a problem hiding this comment.
这条迁移现在语法是 MySQL 兼容的，但它被追加到了已经存在于 main 的 sql/202606.sql。项目文档说明没有自动迁移机制，已有环境按缺失的增量脚本顺序手工执行；已经执行过当前 202606.sql 的环境不会再次执行这个被改写的历史文件，强行重跑还会先在前面的 latest_release_* 字段上遇到重复列。请为本次新增字段放到一个新的增量脚本（或至少独立可重复执行的后续脚本）里，保证已升级到当前 main 的数据库也能应用 openharmony_version。
Co-authored-by: Cursor <[email protected]>
Co-authored-by: Cursor <[email protected]>
Co-authored-by: Cursor <[email protected]>
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
gitcodeOrg: 复用 getValidToken；license 对象归一化；description 对齐模型长度(512)；total_page 兼容大小写并修正注释；handler 增加 org/perPage 输入校验 openharmonyVersion: 区分抓取失败与无匹配版本，失败时保留旧值不误清；全量改为分批游标遍历，避免一次性 load 整个组织进内存 Co-authored-by: Cursor <[email protected]>
2 participants
新增OpenHarmony仓库项目爬取逻辑
可通过REST API 获取组织项目列表获取仓库列表，再通过其他API补充信息
针对OpenHarmony项目，爬取解析适配的鸿蒙系统大版本（大版本只需匹配前两个数字，如v6.0.0.1视为适配v6.0大版本）
可通过gitcode REST API tags获取，规范命名为“OpenHarmony-v6.1-Release”