Add OSM area preflight
This commit is contained in:
@@ -12,6 +12,7 @@
|
||||
| `scripts/build-osm2streets-qgis.js` | intermediates 阶段的实现 | 由 build-area 调起;`npm run build:qgis` 可单跑 |
|
||||
| `scripts/reimport-gpkg.js` | reimport 阶段的实现 | 由 build-area 调起 |
|
||||
| `scripts/diagnose-area.js` | 快速区域诊断;只读配置、OSM 和已有产物 | `npm run diagnose:area` |
|
||||
| `scripts/preflight-area.js` | OSM 构建前预检;只读配置和 OSM,并写成功预检记录 | `npm run preflight:area` |
|
||||
| `scripts/check-area.js` | 区域质量门;复用诊断事实源并给出 PASS/FAIL 退出码 | `npm run check:area` |
|
||||
|
||||
`scripts/parity.js` 和 `scripts/glb-digest.js` 是校验工具,不属于构建链,见
|
||||
@@ -121,6 +122,73 @@ npm run compress:glb -- --input outputs/a/a.glb --output outputs/a/a-compressed-
|
||||
|
||||
---
|
||||
|
||||
## OSM 预检命令
|
||||
|
||||
### 1. Scope / Trigger
|
||||
|
||||
`preflight:area` 用于手工编辑 OSM 后、运行 QGIS / Blender / Cesium 前的快速结构门槛。
|
||||
它不是构建阶段,不进入 `build-area --stages`,不会启动外部工具或写场景产物。
|
||||
|
||||
### 2. Signatures
|
||||
|
||||
```bash
|
||||
npm run preflight:area
|
||||
npm run preflight:area -- --config config/areas/<area>.json
|
||||
```
|
||||
|
||||
### 3. Contracts
|
||||
|
||||
- 通过 `readAreaConfig()` 读取区域配置,只读取配置文件和 OSM XML。
|
||||
- OSM 解析和错误分类来自 `scripts/lib/area-diagnostics.js`,不得另建 XML 解析器。
|
||||
- `<node>`、`<way>`、`<relation>` 带 `action="delete"` 时属于历史删除对象,解析统计和
|
||||
几何检查必须忽略;否则旧 OSM 导出会把已删除的建筑误报为损坏面。
|
||||
- 无效 bounds、任意 way 缺失 node、破损 building way、破损 building multipolygon、
|
||||
不可解析为正米数的显式 building `height` 都是 error,并以非零退出。
|
||||
- 不填 `height` / `building:levels` 合法;无效 `building:levels` 是 warning。
|
||||
- 仅在 error 为零时原子写 `<areaDir>/_pipeline/stages/preflight.manifest.json`。失败不覆盖
|
||||
上次成功 manifest。manifest 记录 config / OSM 文件摘要、预检 summary、errors、warnings
|
||||
和耗时。
|
||||
- `diagnose:area` 显示该 manifest;`check:area` 仅在 manifest 存在时检查其有效性和
|
||||
freshness,故采用前不会使旧区域质量门失败。
|
||||
|
||||
### 4. Validation & Error Matrix
|
||||
|
||||
| 条件 | 结果 |
|
||||
|---|---|
|
||||
| 无效 bounds、缺失 node、破损建筑几何、坏显式 height | 输出 error,非零,不覆盖旧 manifest |
|
||||
| 无效 `building:levels` | 输出 warning,仍可成功写 manifest |
|
||||
| `action="delete"` 的节点 / way / relation | 忽略,不计入当前 OSM 或报错 |
|
||||
| OSM / config 不存在 | `readAreaConfig()` 或文件读取抛错,非零 |
|
||||
|
||||
### 5. Good/Base/Bad Cases
|
||||
|
||||
- Good: 手工编辑后预检 PASS,再运行 `build-area` 的任意重型阶段。
|
||||
- Base: 未运行过预检的旧区域,`check:area` 仍可按既有规则通过。
|
||||
- Bad: 预检失败后仍更新 manifest,或把预检加入 `all` 导致每次构建修改验证记录。
|
||||
|
||||
### 6. Tests Required
|
||||
|
||||
- `npm run test:preflight`
|
||||
- `npm run preflight:area -- --config config/areas/nantaizi-lake-innovation-valley.json`
|
||||
- `npm run diagnose:area -- --config config/areas/nantaizi-lake-innovation-valley.json`
|
||||
- `npm run check:area -- --config config/areas/nantaizi-lake-innovation-valley.json`
|
||||
|
||||
### 7. Wrong vs Correct
|
||||
|
||||
Wrong:
|
||||
|
||||
```js
|
||||
const xml = fs.readFileSync(area.input, "utf8");
|
||||
const missingNodes = xml.match(/<nd[^>]*ref=/g).length;
|
||||
```
|
||||
|
||||
Correct:
|
||||
|
||||
```js
|
||||
const osm = parseOsm(fs.readFileSync(area.input, "utf8"));
|
||||
const preflight = analyzeOsmPreflight(osm);
|
||||
```
|
||||
|
||||
## 区域诊断命令
|
||||
|
||||
### 1. Scope / Trigger
|
||||
@@ -300,8 +368,8 @@ const gate = classifyAreaQuality(result);
|
||||
|
||||
### 1. Scope / Trigger
|
||||
|
||||
Stage manifest 是构建阶段完成后的机器可读产物契约。它覆盖完整区域链:
|
||||
`intermediates`、`reimport`、`blender`、`cesium`、`preview` 和 `compress`。
|
||||
Stage manifest 是区域构建阶段或独立验证通过后的机器可读产物契约。它覆盖预检记录与完整区域链:
|
||||
`preflight`、`intermediates`、`reimport`、`blender`、`cesium`、`preview` 和 `compress`。
|
||||
它用于诊断产物是否存在、是否 stale、体量是否超预算,以及后续 `check:area` /
|
||||
增量构建判断。
|
||||
|
||||
@@ -316,6 +384,7 @@ Manifest 路径固定:
|
||||
当前会写:
|
||||
|
||||
```text
|
||||
<areaDir>/_pipeline/stages/preflight.manifest.json
|
||||
<areaDir>/_pipeline/stages/intermediates.manifest.json
|
||||
<areaDir>/_pipeline/stages/reimport.manifest.json
|
||||
<areaDir>/_pipeline/stages/blender.manifest.json
|
||||
@@ -362,6 +431,15 @@ File records use this shape:
|
||||
}
|
||||
```
|
||||
|
||||
`preflight` manifest:
|
||||
|
||||
- `inputs.config`
|
||||
- `inputs.osm`
|
||||
- `outputs` 固定为空对象(预检不拥有场景产物)
|
||||
- `summary`:bounds、node / way / relation、building way / multipolygon 与问题计数
|
||||
- `errors` 固定为空数组(只有通过的预检会写 manifest)和 `warnings`
|
||||
- manifest 不存在时不算质量门失败;存在但 input 摘要漂移或 JSON 损坏时,`check:area` 失败
|
||||
|
||||
`intermediates` manifest:
|
||||
|
||||
- `inputs.config`
|
||||
|
||||
1
.trellis/tasks/08-04-add-osm-area-preflight/check.jsonl
Normal file
1
.trellis/tasks/08-04-add-osm-area-preflight/check.jsonl
Normal file
@@ -0,0 +1 @@
|
||||
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}
|
||||
30
.trellis/tasks/08-04-add-osm-area-preflight/design.md
Normal file
30
.trellis/tasks/08-04-add-osm-area-preflight/design.md
Normal file
@@ -0,0 +1,30 @@
|
||||
# OSM area preflight design
|
||||
|
||||
## Boundaries
|
||||
|
||||
`scripts/lib/area-diagnostics.js` 保持 OSM 解析和预检分类的唯一事实源。新增 `scripts/preflight-area.js` 只负责 CLI 参数、报告和退出码;它不读取产物,也不调起外部工具。现有 `diagnose-area.js`、`check-area.js` 不复制预检逻辑。
|
||||
|
||||
## Data Flow
|
||||
|
||||
```text
|
||||
area config + OSM XML
|
||||
-> analyzeOsmPreflight()/shared parseOsm()
|
||||
-> { summary, errors, warnings }
|
||||
-> preflight-area CLI report + exit code
|
||||
-> successful preflight.manifest.json
|
||||
-> diagnose/check manifest freshness reporting
|
||||
```
|
||||
|
||||
预检 manifest 是一次成功验证的记录,不是输入所有权或构建依赖。它记录 config/OSM 文件摘要、错误/警告、统计和耗时;预检失败时不写入,也不删除旧 manifest,以保留最后一次成功记录。
|
||||
|
||||
## Classification
|
||||
|
||||
Blocking error 仅限肯定会使场景几何或范围不可靠的 OSM 结构错误:无效 bounds、任意 way 的缺失 node 引用、破损的建筑 way / building multipolygon。没有 `height` / levels 不是错误(渲染已有默认值);填写但不可解析的显式 `height` 是错误。`building:levels` 的非正值或不可解析值将作为 warning,除非现有 Blender 行为表明它会产生错误几何。
|
||||
|
||||
## Compatibility
|
||||
|
||||
现有 `diagnose:area` 保持全量只读报告;它添加 preflight manifest 一行。`check:area` 仅在 manifest 存在时才要求其有效和 fresh,确保现有历史构建仍可通过。新命令的默认配置规则与其他 area CLI 一致。
|
||||
|
||||
## Rollback
|
||||
|
||||
删除 `preflight:area` CLI、预检 manifest 定义和 diagnostics 展示即可回到当前行为。预检 manifest 是可再生文件,保留或删除均不影响构建。
|
||||
@@ -0,0 +1 @@
|
||||
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}
|
||||
23
.trellis/tasks/08-04-add-osm-area-preflight/implement.md
Normal file
23
.trellis/tasks/08-04-add-osm-area-preflight/implement.md
Normal file
@@ -0,0 +1,23 @@
|
||||
# OSM area preflight implementation plan
|
||||
|
||||
1. 确认 Blender 对 `building:levels` 的解析语义;缺失 node 引用按已确认策略一律阻断。
|
||||
2. 在 `area-diagnostics.js` 抽取结构化 OSM 预检分类,并保持 diagnose/check 共用解析。
|
||||
3. 新增 `preflight-area.js` 和 `package.json` 命令;成功后原子写 preflight manifest。
|
||||
4. 扩展 stage manifest 状态与 `diagnose:area` / `check:area` 对 preflight 的显示、freshness 与 failure 规则。
|
||||
5. 为纯 OSM 预检逻辑添加 Node fixture 或等价轻量测试,覆盖关键破损输入。
|
||||
6. 更新 README、pipeline CLI spec、changelog;运行质量检查。
|
||||
|
||||
## Validation
|
||||
|
||||
```bash
|
||||
node --check scripts/lib/area-diagnostics.js
|
||||
node --check scripts/preflight-area.js
|
||||
node --check scripts/diagnose-area.js
|
||||
node --check scripts/check-area.js
|
||||
npm run preflight:area -- --config config/areas/nantaizi-lake-innovation-valley.json
|
||||
npm run diagnose:area -- --config config/areas/nantaizi-lake-innovation-valley.json
|
||||
npm run check:area -- --config config/areas/nantaizi-lake-innovation-valley.json
|
||||
git diff --check
|
||||
```
|
||||
|
||||
只验证 `nantaizi-lake-innovation-valley`。破损 fixture 使用临时目录,不能改写桌面上的 OSM 或正式 outputs。
|
||||
46
.trellis/tasks/08-04-add-osm-area-preflight/prd.md
Normal file
46
.trellis/tasks/08-04-add-osm-area-preflight/prd.md
Normal file
@@ -0,0 +1,46 @@
|
||||
# Add OSM area preflight
|
||||
|
||||
## Goal
|
||||
|
||||
在调用 QGIS、Blender 或 Cesium 前,快速验证区域 OSM 是否包含会导致建筑或场景几何错误的人工编辑问题。使用者应得到明确的通过/失败结果和可定位的问题列表,而不必先跑重型构建阶段。
|
||||
|
||||
## Confirmed Facts
|
||||
|
||||
- `scripts/lib/area-diagnostics.js` 已是区域配置和 OSM 解析的事实源;它已覆盖 bounds、building multipolygon、缺失 node 引用及高度标签,并被 `diagnose:area` 和 `check:area` 共同复用。
|
||||
- `diagnose:area` 是全文诊断,包含既有产物、GLB 与 manifest;它不适合成为编辑 OSM 后的轻量独立门槛。
|
||||
- 当前 `nantaizi-lake-innovation-valley` 是唯一验证目标。忽略 `action=delete` 的历史对象后,其 OSM 有 1 个健康的建筑 multipolygon 和 23 个有效显式高度标签。
|
||||
- 全阶段 manifest 已存在;新预检不应被实现为 `build-area --stages` 的构建阶段。
|
||||
|
||||
## Requirements
|
||||
|
||||
1. 新增 `npm run preflight:area -- --config config/areas/<area>.json`,只读取区域配置和 OSM XML;不得调用 QGIS、Blender、Cesium、压缩或写入场景产物。
|
||||
2. 从共享 diagnostics 模块提供结构化 OSM 预检结果,避免新增第二套 XML/建筑关系解析。
|
||||
3. 预检应检查:有效 bounds、way 引用的 node、building way 闭合性和最小节点数、建筑 multipolygon 的 outer/inner member、成员存在性、可拼接闭环,以及 building `height` / `building:levels` 的基本可解析性。
|
||||
4. 命令输出简短摘要、错误和警告;存在 blocking error 时以非零退出。
|
||||
5. 成功预检写 `<areaDir>/_pipeline/stages/preflight.manifest.json`,记录配置和 OSM 输入的文件摘要、预检摘要、errors 与 warnings。失败预检不得覆盖此前成功 manifest。
|
||||
6. `diagnose:area` 显示预检 manifest 的状态;`check:area` 仅在该 manifest 已存在时检查其新鲜度和有效性,避免使历史区域在首次采用前立即失败。
|
||||
7. README 和 Pipeline spec 记录命令、退出码与 manifest 所有权。
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- [x] 当前南台子 OSM 上 `npm run preflight:area -- --config config/areas/nantaizi-lake-innovation-valley.json` 输出 PASS、写入 fresh `preflight.manifest.json`,且不触发重型工具。
|
||||
- [x] 预检报告包含 buildings、multipolygon、bounds 和几何健康统计。
|
||||
- [x] 通过临时 OSM fixture 验证:缺失/无效 bounds、缺失 node、未闭合 building way、破损 building multipolygon、无效显式 height 都能得到定位明确的 blocking error。
|
||||
- [x] `diagnose:area` 显示 preflight manifest,`check:area` 对存在但 stale/invalid 的 preflight manifest 失败。
|
||||
- [x] 原有 `diagnose:area` 与 `check:area` 继续通过当前南台子区域。
|
||||
- [x] 修改脚本通过 Node 语法检查和 `git diff --check`。
|
||||
|
||||
## Out Of Scope
|
||||
|
||||
- 不修复 OSM 文件,不自动重组关系或补齐标签。
|
||||
- 不检查道路拓扑、转向箭头渲染、osm2streets 的语义兼容性,或任意非建筑面是否合法。
|
||||
- 不把预检加入 `build-area` 的默认阶段,也不要求未运行过预检的历史产物立即失败。
|
||||
- 不验证 `hanyang-block`。
|
||||
|
||||
## Key Decisions
|
||||
|
||||
- 缺失 node 引用一律作为 blocking error。它表示 OSM 引用完整性已损坏,即使当前出现在非建筑 way,也不将问题带入后续构建。
|
||||
|
||||
## Notes
|
||||
|
||||
- 这是跨 CLI、OSM 解析、质量门和 manifest 的改动,按复杂任务处理,需要 design 和 implement 工件后再开始实现。
|
||||
26
.trellis/tasks/08-04-add-osm-area-preflight/task.json
Normal file
26
.trellis/tasks/08-04-add-osm-area-preflight/task.json
Normal file
@@ -0,0 +1,26 @@
|
||||
{
|
||||
"id": "add-osm-area-preflight",
|
||||
"name": "add-osm-area-preflight",
|
||||
"title": "Add OSM area preflight",
|
||||
"description": "",
|
||||
"status": "in_progress",
|
||||
"dev_type": null,
|
||||
"scope": null,
|
||||
"package": null,
|
||||
"priority": "P2",
|
||||
"creator": "dingkang",
|
||||
"assignee": "dingkang",
|
||||
"createdAt": "2026-08-04",
|
||||
"completedAt": null,
|
||||
"branch": null,
|
||||
"base_branch": "main",
|
||||
"worktree_path": null,
|
||||
"commit": null,
|
||||
"pr_url": null,
|
||||
"subtasks": [],
|
||||
"children": [],
|
||||
"parent": null,
|
||||
"relatedFiles": [],
|
||||
"notes": "",
|
||||
"meta": {}
|
||||
}
|
||||
14
README.md
14
README.md
@@ -72,13 +72,25 @@ npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json
|
||||
npm run diagnose:area -- --config config/areas/nantaizi-lake-innovation-valley.json
|
||||
```
|
||||
|
||||
每次手工编辑 OSM 后、进入重型构建前,先跑预检。它只读取区域配置和 OSM XML,检查
|
||||
bounds、缺失 node 引用、building way 闭合性、building multipolygon member/ring、以及
|
||||
显式 `height` / `building:levels` 标签:
|
||||
|
||||
```bash
|
||||
npm run preflight:area -- --config config/areas/nantaizi-lake-innovation-valley.json
|
||||
```
|
||||
|
||||
预检有 error 时退出非零且不会更新记录;通过后写 `preflight.manifest.json`,保留本次
|
||||
验证的 config / OSM 文件摘要和检查结果。
|
||||
|
||||
诊断会输出 OSM bounds、building way / multipolygon relation、显式高度、植被数量、
|
||||
现有产物状态,以及 GLB 的 size / nodes / meshes / materials / images / extensions。
|
||||
缺少已期望的基线产物、异常 building relation、GLB 超过保守预算等会进入 `Warnings`。
|
||||
|
||||
每个成功的区域构建阶段都会写机器可读的 stage manifest:
|
||||
预检和每个成功的区域构建阶段都会写机器可读的 manifest:
|
||||
|
||||
```text
|
||||
outputs/<area-id>/_pipeline/stages/preflight.manifest.json
|
||||
outputs/<area-id>/_pipeline/stages/intermediates.manifest.json
|
||||
outputs/<area-id>/_pipeline/stages/reimport.manifest.json
|
||||
outputs/<area-id>/_pipeline/stages/blender.manifest.json
|
||||
|
||||
@@ -2,6 +2,11 @@
|
||||
|
||||
## 2026-08-04
|
||||
|
||||
- 新增 `npm run preflight:area -- --config config/areas/<area>.json`:手工编辑 OSM 后可在
|
||||
QGIS / Blender / Cesium 前检查 bounds、缺失 node、building way 闭合性、building
|
||||
multipolygon member/ring 与显式高度。结构错误会非零退出;成功结果原子写入
|
||||
`_pipeline/stages/preflight.manifest.json`,并由 `diagnose:area` / `check:area` 检查
|
||||
新鲜度。该命令不属于 `build-area` 阶段,也不调用重型工具。
|
||||
- 扩展 stage manifest 到完整区域链:`intermediates`、`reimport`、`blender` 和
|
||||
`preview` 现在和既有 `cesium` / `compress` 一样,在成功后写
|
||||
`_pipeline/stages/<stage>.manifest.json`。前段 manifest 记录 config / OSM /
|
||||
|
||||
@@ -9,7 +9,9 @@
|
||||
"build:qgis": "node scripts/build-osm2streets-qgis.js",
|
||||
"check:area": "node scripts/check-area.js",
|
||||
"compress:glb": "node scripts/compress-glb.js",
|
||||
"diagnose:area": "node scripts/diagnose-area.js"
|
||||
"diagnose:area": "node scripts/diagnose-area.js",
|
||||
"preflight:area": "node scripts/preflight-area.js",
|
||||
"test:preflight": "node scripts/test-area-preflight.js"
|
||||
},
|
||||
"dependencies": {
|
||||
"osm2streets-js-node": "0.1.4"
|
||||
|
||||
@@ -67,6 +67,7 @@ function parseOsm(xml) {
|
||||
const nodePattern = /<node\b([^>]*?)\/>|<node\b([^>]*?)>([\s\S]*?)<\/node>/g;
|
||||
for (const match of xml.matchAll(nodePattern)) {
|
||||
const attrs = xmlAttrs(match[1] || match[2] || "");
|
||||
if (attrs.action === "delete") continue;
|
||||
if (attrs.id) nodeIds.add(attrs.id);
|
||||
nodeStats.total += 1;
|
||||
const tags = parseTags(match[3] || "");
|
||||
@@ -80,6 +81,8 @@ function parseOsm(xml) {
|
||||
buildingsWithHeight: 0,
|
||||
buildingsWithLevels: 0,
|
||||
buildingsWithBadHeight: 0,
|
||||
buildingsWithBadLevels: 0,
|
||||
buildingGeometryIssues: [],
|
||||
missingNodeRefs: 0,
|
||||
grass: 0,
|
||||
scrub: 0,
|
||||
@@ -87,6 +90,7 @@ function parseOsm(xml) {
|
||||
};
|
||||
for (const match of xml.matchAll(/<way\b([^>]*)>([\s\S]*?)<\/way>/g)) {
|
||||
const attrs = xmlAttrs(match[1]);
|
||||
if (attrs.action === "delete") continue;
|
||||
const body = match[2];
|
||||
const tags = parseTags(body);
|
||||
const refs = [];
|
||||
@@ -110,6 +114,14 @@ function parseOsm(xml) {
|
||||
if (isExplicitHeight(tags)) wayStats.buildingsWithHeight += 1;
|
||||
if (tags["building:levels"]) wayStats.buildingsWithLevels += 1;
|
||||
if (tags.height && !parseHeightMeters(tags.height)) wayStats.buildingsWithBadHeight += 1;
|
||||
if (tags["building:levels"] && !parseBuildingLevels(tags["building:levels"])) {
|
||||
wayStats.buildingsWithBadLevels += 1;
|
||||
}
|
||||
if (refs.length < 4) {
|
||||
wayStats.buildingGeometryIssues.push({ id: way.id, issue: "has fewer than 4 node refs" });
|
||||
} else if (!way.closed) {
|
||||
wayStats.buildingGeometryIssues.push({ id: way.id, issue: "is not closed" });
|
||||
}
|
||||
}
|
||||
if (tags.landuse === "grass") wayStats.grass += 1;
|
||||
if (tags.natural === "scrub") wayStats.scrub += 1;
|
||||
@@ -122,11 +134,13 @@ function parseOsm(xml) {
|
||||
buildingsWithHeight: 0,
|
||||
buildingsWithLevels: 0,
|
||||
buildingsWithBadHeight: 0,
|
||||
buildingsWithBadLevels: 0,
|
||||
healthyBuildingMultipolygons: 0,
|
||||
issues: [],
|
||||
};
|
||||
for (const match of xml.matchAll(/<relation\b([^>]*)>([\s\S]*?)<\/relation>/g)) {
|
||||
const attrs = xmlAttrs(match[1]);
|
||||
if (attrs.action === "delete") continue;
|
||||
const body = match[2];
|
||||
const tags = parseTags(body);
|
||||
const members = [];
|
||||
@@ -139,6 +153,9 @@ function parseOsm(xml) {
|
||||
if (isExplicitHeight(tags)) relationStats.buildingsWithHeight += 1;
|
||||
if (tags["building:levels"]) relationStats.buildingsWithLevels += 1;
|
||||
if (tags.height && !parseHeightMeters(tags.height)) relationStats.buildingsWithBadHeight += 1;
|
||||
if (tags["building:levels"] && !parseBuildingLevels(tags["building:levels"])) {
|
||||
relationStats.buildingsWithBadLevels += 1;
|
||||
}
|
||||
|
||||
const health = buildingRelationHealth(attrs.id || "", members, ways);
|
||||
if (health.ok) relationStats.healthyBuildingMultipolygons += 1;
|
||||
@@ -163,7 +180,10 @@ function parseBounds(xml) {
|
||||
maxLon: Number(attrs.maxlon),
|
||||
maxLat: Number(attrs.maxlat),
|
||||
};
|
||||
return Object.values(bounds).every(Number.isFinite) ? bounds : null;
|
||||
return Object.values(bounds).every(Number.isFinite) &&
|
||||
bounds.minLon < bounds.maxLon && bounds.minLat < bounds.maxLat
|
||||
? bounds
|
||||
: null;
|
||||
}
|
||||
|
||||
function isExplicitHeight(tags) {
|
||||
@@ -177,6 +197,48 @@ function parseHeightMeters(value) {
|
||||
return Number.isFinite(height) && height > 0 ? height : null;
|
||||
}
|
||||
|
||||
function parseBuildingLevels(value) {
|
||||
const levels = Number(String(value).trim());
|
||||
return Number.isFinite(levels) && levels > 0;
|
||||
}
|
||||
|
||||
function analyzeOsmPreflight(osm) {
|
||||
const errors = [];
|
||||
const warnings = [];
|
||||
if (!osm.bounds) errors.push("OSM has no valid <bounds>.");
|
||||
if (osm.ways.missingNodeRefs) {
|
||||
errors.push(`OSM ways reference ${osm.ways.missingNodeRefs} missing node(s).`);
|
||||
}
|
||||
for (const issue of osm.ways.buildingGeometryIssues) {
|
||||
errors.push(`Building way ${issue.id}: ${issue.issue}.`);
|
||||
}
|
||||
if (osm.ways.buildingsWithBadHeight || osm.relations.buildingsWithBadHeight) {
|
||||
errors.push("Some building height tags could not be parsed as positive meters.");
|
||||
}
|
||||
for (const issue of osm.relations.issues) {
|
||||
errors.push(`Building relation ${issue.id}: ${issue.issues.join("; ")}.`);
|
||||
}
|
||||
const badLevels = osm.ways.buildingsWithBadLevels + osm.relations.buildingsWithBadLevels;
|
||||
if (badLevels) {
|
||||
warnings.push(`${badLevels} building:levels tag(s) could not be parsed as positive numbers.`);
|
||||
}
|
||||
return {
|
||||
errors: uniqueLines(errors),
|
||||
warnings: uniqueLines(warnings),
|
||||
summary: {
|
||||
bounds: osm.bounds,
|
||||
nodes: osm.nodes.total,
|
||||
ways: osm.ways.total,
|
||||
relations: osm.relations.total,
|
||||
buildingWays: osm.ways.buildings,
|
||||
buildingMultipolygons: osm.relations.buildingMultipolygons,
|
||||
missingNodeRefs: osm.ways.missingNodeRefs,
|
||||
buildingWayIssues: osm.ways.buildingGeometryIssues.length,
|
||||
buildingRelationIssues: osm.relations.issues.length,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
function buildingRelationHealth(id, members, ways) {
|
||||
const issues = [];
|
||||
const outerMembers = members.filter((member) => member.type === "way" && member.role === "outer");
|
||||
@@ -307,6 +369,15 @@ function stageManifestStatus(area, configPath = null) {
|
||||
const hasReimportManifest = fs.existsSync(reimportManifest);
|
||||
const derivedConfig = path.join(area.outputs.pipelineDir, "osm2streets-qgis.config.json");
|
||||
const stages = [
|
||||
{
|
||||
stage: "preflight",
|
||||
expected: fs.existsSync(stageManifestPath(area, "preflight")),
|
||||
inputs: {
|
||||
...(configPath ? { config: configPath } : {}),
|
||||
osm: area.input,
|
||||
},
|
||||
outputs: {},
|
||||
},
|
||||
{
|
||||
stage: "intermediates",
|
||||
expected: (
|
||||
@@ -763,6 +834,7 @@ function formatBytes(bytes) {
|
||||
|
||||
module.exports = {
|
||||
BUDGETS,
|
||||
analyzeOsmPreflight,
|
||||
analyzeArea,
|
||||
artifactStatus,
|
||||
classifyAreaQuality,
|
||||
|
||||
99
scripts/preflight-area.js
Normal file
99
scripts/preflight-area.js
Normal file
@@ -0,0 +1,99 @@
|
||||
#!/usr/bin/env node
|
||||
"use strict";
|
||||
|
||||
const fs = require("fs");
|
||||
const path = require("path");
|
||||
const { readAreaConfig } = require("./lib/area-config");
|
||||
const {
|
||||
analyzeOsmPreflight,
|
||||
defaultConfigPath,
|
||||
parseOsm,
|
||||
} = require("./lib/area-diagnostics");
|
||||
const { fileRecord, writeStageManifest } = require("./lib/stage-manifest");
|
||||
|
||||
const repoRoot = path.resolve(__dirname, "..");
|
||||
|
||||
function parseArgs(argv) {
|
||||
const out = {};
|
||||
for (let i = 0; i < argv.length; i += 1) {
|
||||
const arg = argv[i];
|
||||
if (!arg.startsWith("--")) continue;
|
||||
const key = arg.slice(2).replace(/-([a-z])/g, (_, c) => c.toUpperCase());
|
||||
const next = argv[i + 1];
|
||||
if (!next || next.startsWith("--")) {
|
||||
out[key] = "true";
|
||||
} else {
|
||||
out[key] = next;
|
||||
i += 1;
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function main() {
|
||||
const startedAt = new Date().toISOString();
|
||||
const args = parseArgs(process.argv.slice(2));
|
||||
const configPath = path.resolve(args.config || defaultConfigPath(repoRoot));
|
||||
const area = readAreaConfig(configPath, { repoRoot });
|
||||
const osm = parseOsm(fs.readFileSync(area.input, "utf8"));
|
||||
const preflight = analyzeOsmPreflight(osm);
|
||||
|
||||
printPreflightReport(area, configPath, preflight);
|
||||
if (preflight.errors.length) {
|
||||
process.exitCode = 1;
|
||||
return;
|
||||
}
|
||||
|
||||
const finishedAt = new Date().toISOString();
|
||||
writeStageManifest(area, {
|
||||
stage: "preflight",
|
||||
status: "ok",
|
||||
config: configPath,
|
||||
startedAt,
|
||||
finishedAt,
|
||||
durationMs: Date.parse(finishedAt) - Date.parse(startedAt),
|
||||
inputs: {
|
||||
config: fileRecord(configPath),
|
||||
osm: fileRecord(area.input),
|
||||
},
|
||||
outputs: {},
|
||||
summary: preflight.summary,
|
||||
errors: preflight.errors,
|
||||
warnings: preflight.warnings,
|
||||
});
|
||||
}
|
||||
|
||||
function printPreflightReport(area, configPath, preflight) {
|
||||
console.log("OSM area preflight");
|
||||
console.log(`Area: ${area.id}`);
|
||||
console.log(`Config: ${configPath}`);
|
||||
console.log(`Input: ${area.input}`);
|
||||
console.log("");
|
||||
console.log(
|
||||
`OSM: ${preflight.summary.nodes} nodes, ${preflight.summary.ways} ways, ` +
|
||||
`${preflight.summary.relations} relations`,
|
||||
);
|
||||
console.log(
|
||||
`Buildings: ${preflight.summary.buildingWays} way(s), ` +
|
||||
`${preflight.summary.buildingMultipolygons} multipolygon relation(s)`,
|
||||
);
|
||||
console.log(`Bounds: ${preflight.summary.bounds ? "valid" : "invalid or missing"}`);
|
||||
console.log("");
|
||||
const status = preflight.errors.length ? "FAIL" : "PASS";
|
||||
console.log(`${status}: ${preflight.errors.length} error(s), ${preflight.warnings.length} warning(s)`);
|
||||
console.log("");
|
||||
printLines("Errors", preflight.errors);
|
||||
console.log("");
|
||||
printLines("Warnings", preflight.warnings);
|
||||
}
|
||||
|
||||
function printLines(label, lines) {
|
||||
console.log(`${label} (${lines.length})`);
|
||||
if (!lines.length) {
|
||||
console.log(" none");
|
||||
return;
|
||||
}
|
||||
for (const line of lines) console.log(` - ${line}`);
|
||||
}
|
||||
|
||||
main();
|
||||
34
scripts/test-area-preflight.js
Normal file
34
scripts/test-area-preflight.js
Normal file
@@ -0,0 +1,34 @@
|
||||
#!/usr/bin/env node
|
||||
"use strict";
|
||||
|
||||
const assert = require("assert");
|
||||
const { analyzeOsmPreflight, parseOsm } = require("./lib/area-diagnostics");
|
||||
|
||||
function osm({ bounds = '<bounds minlon="0" minlat="0" maxlon="1" maxlat="1"/>', nodes, ways, relations = "" }) {
|
||||
return `<osm>${bounds}${nodes}${ways}${relations}</osm>`;
|
||||
}
|
||||
|
||||
function nodes(ids) {
|
||||
return ids.map((id) => `<node id="${id}" lat="0" lon="0"/>`).join("");
|
||||
}
|
||||
|
||||
function way(id, refs, tags = '<tag k="building" v="yes"/>') {
|
||||
return `<way id="${id}">${refs.map((ref) => `<nd ref="${ref}"/>`).join("")}${tags}</way>`;
|
||||
}
|
||||
|
||||
function preflight(input) {
|
||||
return analyzeOsmPreflight(parseOsm(input));
|
||||
}
|
||||
|
||||
assert.equal(preflight(osm({ nodes: nodes([1, 2, 3]), ways: way(10, [1, 2, 3, 1]) })).errors.length, 0);
|
||||
assert.match(preflight(osm({ bounds: "", nodes: nodes([1, 2, 3]), ways: way(10, [1, 2, 3, 1]) })).errors.join("\n"), /bounds/);
|
||||
assert.match(preflight(osm({ bounds: '<bounds minlon="1" minlat="0" maxlon="0" maxlat="1"/>', nodes: nodes([1, 2, 3]), ways: way(10, [1, 2, 3, 1]) })).errors.join("\n"), /bounds/);
|
||||
assert.match(preflight(osm({ nodes: nodes([1, 2, 3]), ways: way(10, [1, 2, 9, 1]) })).errors.join("\n"), /missing node/);
|
||||
assert.match(preflight(osm({ nodes: nodes([1, 2, 3]), ways: way(10, [1, 2, 3, 2]) })).errors.join("\n"), /not closed/);
|
||||
assert.match(preflight(osm({ nodes: nodes([1, 2, 3]), ways: way(10, [1, 2, 3, 1], '<tag k="building" v="yes"/><tag k="height" v="many"/>') })).errors.join("\n"), /height/);
|
||||
assert.equal(preflight(osm({ nodes: nodes([1, 2, 3]), ways: '<way id="10" action="delete"><tag k="building" v="yes"/></way>' })).errors.length, 0);
|
||||
|
||||
const brokenRelation = '<relation id="20"><member type="way" ref="99" role="outer"/><tag k="type" v="multipolygon"/><tag k="building" v="yes"/></relation>';
|
||||
assert.match(preflight(osm({ nodes: nodes([1, 2, 3]), ways: "", relations: brokenRelation })).errors.join("\n"), /Building relation 20/);
|
||||
|
||||
console.log("OSM preflight tests passed.");
|
||||
Reference in New Issue
Block a user