Add OSM area preflight

This commit is contained in:
2026-08-04 11:58:12 +08:00
parent 34153a31ab
commit e1f5207e97
13 changed files with 434 additions and 5 deletions

View File

@@ -12,6 +12,7 @@
| `scripts/build-osm2streets-qgis.js` | intermediates 阶段的实现 | 由 build-area 调起;`npm run build:qgis` 可单跑 | | `scripts/build-osm2streets-qgis.js` | intermediates 阶段的实现 | 由 build-area 调起;`npm run build:qgis` 可单跑 |
| `scripts/reimport-gpkg.js` | reimport 阶段的实现 | 由 build-area 调起 | | `scripts/reimport-gpkg.js` | reimport 阶段的实现 | 由 build-area 调起 |
| `scripts/diagnose-area.js` | 快速区域诊断只读配置、OSM 和已有产物 | `npm run diagnose:area` | | `scripts/diagnose-area.js` | 快速区域诊断只读配置、OSM 和已有产物 | `npm run diagnose:area` |
| `scripts/preflight-area.js` | OSM 构建前预检;只读配置和 OSM并写成功预检记录 | `npm run preflight:area` |
| `scripts/check-area.js` | 区域质量门;复用诊断事实源并给出 PASS/FAIL 退出码 | `npm run check:area` | | `scripts/check-area.js` | 区域质量门;复用诊断事实源并给出 PASS/FAIL 退出码 | `npm run check:area` |
`scripts/parity.js``scripts/glb-digest.js` 是校验工具,不属于构建链,见 `scripts/parity.js``scripts/glb-digest.js` 是校验工具,不属于构建链,见
@@ -121,6 +122,73 @@ npm run compress:glb -- --input outputs/a/a.glb --output outputs/a/a-compressed-
--- ---
## OSM 预检命令
### 1. Scope / Trigger
`preflight:area` 用于手工编辑 OSM 后、运行 QGIS / Blender / Cesium 前的快速结构门槛。
它不是构建阶段,不进入 `build-area --stages`,不会启动外部工具或写场景产物。
### 2. Signatures
```bash
npm run preflight:area
npm run preflight:area -- --config config/areas/<area>.json
```
### 3. Contracts
- 通过 `readAreaConfig()` 读取区域配置,只读取配置文件和 OSM XML。
- OSM 解析和错误分类来自 `scripts/lib/area-diagnostics.js`,不得另建 XML 解析器。
- `<node>``<way>``<relation>``action="delete"` 时属于历史删除对象,解析统计和
几何检查必须忽略;否则旧 OSM 导出会把已删除的建筑误报为损坏面。
- 无效 bounds、任意 way 缺失 node、破损 building way、破损 building multipolygon、
不可解析为正米数的显式 building `height` 都是 error并以非零退出。
- 不填 `height` / `building:levels` 合法;无效 `building:levels` 是 warning。
- 仅在 error 为零时原子写 `<areaDir>/_pipeline/stages/preflight.manifest.json`。失败不覆盖
上次成功 manifest。manifest 记录 config / OSM 文件摘要、预检 summary、errors、warnings
和耗时。
- `diagnose:area` 显示该 manifest`check:area` 仅在 manifest 存在时检查其有效性和
freshness故采用前不会使旧区域质量门失败。
### 4. Validation & Error Matrix
| 条件 | 结果 |
|---|---|
| 无效 bounds、缺失 node、破损建筑几何、坏显式 height | 输出 error非零不覆盖旧 manifest |
| 无效 `building:levels` | 输出 warning仍可成功写 manifest |
| `action="delete"` 的节点 / way / relation | 忽略,不计入当前 OSM 或报错 |
| OSM / config 不存在 | `readAreaConfig()` 或文件读取抛错,非零 |
### 5. Good/Base/Bad Cases
- Good: 手工编辑后预检 PASS再运行 `build-area` 的任意重型阶段。
- Base: 未运行过预检的旧区域,`check:area` 仍可按既有规则通过。
- Bad: 预检失败后仍更新 manifest或把预检加入 `all` 导致每次构建修改验证记录。
### 6. Tests Required
- `npm run test:preflight`
- `npm run preflight:area -- --config config/areas/nantaizi-lake-innovation-valley.json`
- `npm run diagnose:area -- --config config/areas/nantaizi-lake-innovation-valley.json`
- `npm run check:area -- --config config/areas/nantaizi-lake-innovation-valley.json`
### 7. Wrong vs Correct
Wrong:
```js
const xml = fs.readFileSync(area.input, "utf8");
const missingNodes = xml.match(/<nd[^>]*ref=/g).length;
```
Correct:
```js
const osm = parseOsm(fs.readFileSync(area.input, "utf8"));
const preflight = analyzeOsmPreflight(osm);
```
## 区域诊断命令 ## 区域诊断命令
### 1. Scope / Trigger ### 1. Scope / Trigger
@@ -300,8 +368,8 @@ const gate = classifyAreaQuality(result);
### 1. Scope / Trigger ### 1. Scope / Trigger
Stage manifest 是构建阶段完成后的机器可读产物契约。它覆盖完整区域链: Stage manifest 是区域构建阶段或独立验证通过后的机器可读产物契约。它覆盖预检记录与完整区域链:
`intermediates``reimport``blender``cesium``preview``compress` `preflight``intermediates``reimport``blender``cesium``preview``compress`
它用于诊断产物是否存在、是否 stale、体量是否超预算以及后续 `check:area` / 它用于诊断产物是否存在、是否 stale、体量是否超预算以及后续 `check:area` /
增量构建判断。 增量构建判断。
@@ -316,6 +384,7 @@ Manifest 路径固定:
当前会写: 当前会写:
```text ```text
<areaDir>/_pipeline/stages/preflight.manifest.json
<areaDir>/_pipeline/stages/intermediates.manifest.json <areaDir>/_pipeline/stages/intermediates.manifest.json
<areaDir>/_pipeline/stages/reimport.manifest.json <areaDir>/_pipeline/stages/reimport.manifest.json
<areaDir>/_pipeline/stages/blender.manifest.json <areaDir>/_pipeline/stages/blender.manifest.json
@@ -362,6 +431,15 @@ File records use this shape:
} }
``` ```
`preflight` manifest:
- `inputs.config`
- `inputs.osm`
- `outputs` 固定为空对象(预检不拥有场景产物)
- `summary`bounds、node / way / relation、building way / multipolygon 与问题计数
- `errors` 固定为空数组(只有通过的预检会写 manifest`warnings`
- manifest 不存在时不算质量门失败;存在但 input 摘要漂移或 JSON 损坏时,`check:area` 失败
`intermediates` manifest: `intermediates` manifest:
- `inputs.config` - `inputs.config`

View File

@@ -0,0 +1 @@
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}

View File

@@ -0,0 +1,30 @@
# OSM area preflight design
## Boundaries
`scripts/lib/area-diagnostics.js` 保持 OSM 解析和预检分类的唯一事实源。新增 `scripts/preflight-area.js` 只负责 CLI 参数、报告和退出码;它不读取产物,也不调起外部工具。现有 `diagnose-area.js``check-area.js` 不复制预检逻辑。
## Data Flow
```text
area config + OSM XML
-> analyzeOsmPreflight()/shared parseOsm()
-> { summary, errors, warnings }
-> preflight-area CLI report + exit code
-> successful preflight.manifest.json
-> diagnose/check manifest freshness reporting
```
预检 manifest 是一次成功验证的记录,不是输入所有权或构建依赖。它记录 config/OSM 文件摘要、错误/警告、统计和耗时;预检失败时不写入,也不删除旧 manifest以保留最后一次成功记录。
## Classification
Blocking error 仅限肯定会使场景几何或范围不可靠的 OSM 结构错误:无效 bounds、任意 way 的缺失 node 引用、破损的建筑 way / building multipolygon。没有 `height` / levels 不是错误(渲染已有默认值);填写但不可解析的显式 `height` 是错误。`building:levels` 的非正值或不可解析值将作为 warning除非现有 Blender 行为表明它会产生错误几何。
## Compatibility
现有 `diagnose:area` 保持全量只读报告;它添加 preflight manifest 一行。`check:area` 仅在 manifest 存在时才要求其有效和 fresh确保现有历史构建仍可通过。新命令的默认配置规则与其他 area CLI 一致。
## Rollback
删除 `preflight:area` CLI、预检 manifest 定义和 diagnostics 展示即可回到当前行为。预检 manifest 是可再生文件,保留或删除均不影响构建。

View File

@@ -0,0 +1 @@
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}

View File

@@ -0,0 +1,23 @@
# OSM area preflight implementation plan
1. 确认 Blender 对 `building:levels` 的解析语义;缺失 node 引用按已确认策略一律阻断。
2.`area-diagnostics.js` 抽取结构化 OSM 预检分类,并保持 diagnose/check 共用解析。
3. 新增 `preflight-area.js``package.json` 命令;成功后原子写 preflight manifest。
4. 扩展 stage manifest 状态与 `diagnose:area` / `check:area` 对 preflight 的显示、freshness 与 failure 规则。
5. 为纯 OSM 预检逻辑添加 Node fixture 或等价轻量测试,覆盖关键破损输入。
6. 更新 README、pipeline CLI spec、changelog运行质量检查。
## Validation
```bash
node --check scripts/lib/area-diagnostics.js
node --check scripts/preflight-area.js
node --check scripts/diagnose-area.js
node --check scripts/check-area.js
npm run preflight:area -- --config config/areas/nantaizi-lake-innovation-valley.json
npm run diagnose:area -- --config config/areas/nantaizi-lake-innovation-valley.json
npm run check:area -- --config config/areas/nantaizi-lake-innovation-valley.json
git diff --check
```
只验证 `nantaizi-lake-innovation-valley`。破损 fixture 使用临时目录,不能改写桌面上的 OSM 或正式 outputs。

View File

@@ -0,0 +1,46 @@
# Add OSM area preflight
## Goal
在调用 QGIS、Blender 或 Cesium 前,快速验证区域 OSM 是否包含会导致建筑或场景几何错误的人工编辑问题。使用者应得到明确的通过/失败结果和可定位的问题列表,而不必先跑重型构建阶段。
## Confirmed Facts
- `scripts/lib/area-diagnostics.js` 已是区域配置和 OSM 解析的事实源;它已覆盖 bounds、building multipolygon、缺失 node 引用及高度标签,并被 `diagnose:area``check:area` 共同复用。
- `diagnose:area` 是全文诊断包含既有产物、GLB 与 manifest它不适合成为编辑 OSM 后的轻量独立门槛。
- 当前 `nantaizi-lake-innovation-valley` 是唯一验证目标。忽略 `action=delete` 的历史对象后,其 OSM 有 1 个健康的建筑 multipolygon 和 23 个有效显式高度标签。
- 全阶段 manifest 已存在;新预检不应被实现为 `build-area --stages` 的构建阶段。
## Requirements
1. 新增 `npm run preflight:area -- --config config/areas/<area>.json`,只读取区域配置和 OSM XML不得调用 QGIS、Blender、Cesium、压缩或写入场景产物。
2. 从共享 diagnostics 模块提供结构化 OSM 预检结果,避免新增第二套 XML/建筑关系解析。
3. 预检应检查:有效 bounds、way 引用的 node、building way 闭合性和最小节点数、建筑 multipolygon 的 outer/inner member、成员存在性、可拼接闭环以及 building `height` / `building:levels` 的基本可解析性。
4. 命令输出简短摘要、错误和警告;存在 blocking error 时以非零退出。
5. 成功预检写 `<areaDir>/_pipeline/stages/preflight.manifest.json`,记录配置和 OSM 输入的文件摘要、预检摘要、errors 与 warnings。失败预检不得覆盖此前成功 manifest。
6. `diagnose:area` 显示预检 manifest 的状态;`check:area` 仅在该 manifest 已存在时检查其新鲜度和有效性,避免使历史区域在首次采用前立即失败。
7. README 和 Pipeline spec 记录命令、退出码与 manifest 所有权。
## Acceptance Criteria
- [x] 当前南台子 OSM 上 `npm run preflight:area -- --config config/areas/nantaizi-lake-innovation-valley.json` 输出 PASS、写入 fresh `preflight.manifest.json`,且不触发重型工具。
- [x] 预检报告包含 buildings、multipolygon、bounds 和几何健康统计。
- [x] 通过临时 OSM fixture 验证:缺失/无效 bounds、缺失 node、未闭合 building way、破损 building multipolygon、无效显式 height 都能得到定位明确的 blocking error。
- [x] `diagnose:area` 显示 preflight manifest`check:area` 对存在但 stale/invalid 的 preflight manifest 失败。
- [x] 原有 `diagnose:area``check:area` 继续通过当前南台子区域。
- [x] 修改脚本通过 Node 语法检查和 `git diff --check`
## Out Of Scope
- 不修复 OSM 文件,不自动重组关系或补齐标签。
- 不检查道路拓扑、转向箭头渲染、osm2streets 的语义兼容性,或任意非建筑面是否合法。
- 不把预检加入 `build-area` 的默认阶段,也不要求未运行过预检的历史产物立即失败。
- 不验证 `hanyang-block`
## Key Decisions
- 缺失 node 引用一律作为 blocking error。它表示 OSM 引用完整性已损坏,即使当前出现在非建筑 way也不将问题带入后续构建。
## Notes
- 这是跨 CLI、OSM 解析、质量门和 manifest 的改动,按复杂任务处理,需要 design 和 implement 工件后再开始实现。

View File

@@ -0,0 +1,26 @@
{
"id": "add-osm-area-preflight",
"name": "add-osm-area-preflight",
"title": "Add OSM area preflight",
"description": "",
"status": "in_progress",
"dev_type": null,
"scope": null,
"package": null,
"priority": "P2",
"creator": "dingkang",
"assignee": "dingkang",
"createdAt": "2026-08-04",
"completedAt": null,
"branch": null,
"base_branch": "main",
"worktree_path": null,
"commit": null,
"pr_url": null,
"subtasks": [],
"children": [],
"parent": null,
"relatedFiles": [],
"notes": "",
"meta": {}
}

View File

@@ -72,13 +72,25 @@ npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json
npm run diagnose:area -- --config config/areas/nantaizi-lake-innovation-valley.json npm run diagnose:area -- --config config/areas/nantaizi-lake-innovation-valley.json
``` ```
每次手工编辑 OSM 后、进入重型构建前,先跑预检。它只读取区域配置和 OSM XML检查
bounds、缺失 node 引用、building way 闭合性、building multipolygon member/ring、以及
显式 `height` / `building:levels` 标签:
```bash
npm run preflight:area -- --config config/areas/nantaizi-lake-innovation-valley.json
```
预检有 error 时退出非零且不会更新记录;通过后写 `preflight.manifest.json`,保留本次
验证的 config / OSM 文件摘要和检查结果。
诊断会输出 OSM bounds、building way / multipolygon relation、显式高度、植被数量、 诊断会输出 OSM bounds、building way / multipolygon relation、显式高度、植被数量、
现有产物状态,以及 GLB 的 size / nodes / meshes / materials / images / extensions。 现有产物状态,以及 GLB 的 size / nodes / meshes / materials / images / extensions。
缺少已期望的基线产物、异常 building relation、GLB 超过保守预算等会进入 `Warnings` 缺少已期望的基线产物、异常 building relation、GLB 超过保守预算等会进入 `Warnings`
每个成功的区域构建阶段都会写机器可读的 stage manifest 预检和每个成功的区域构建阶段都会写机器可读的 manifest
```text ```text
outputs/<area-id>/_pipeline/stages/preflight.manifest.json
outputs/<area-id>/_pipeline/stages/intermediates.manifest.json outputs/<area-id>/_pipeline/stages/intermediates.manifest.json
outputs/<area-id>/_pipeline/stages/reimport.manifest.json outputs/<area-id>/_pipeline/stages/reimport.manifest.json
outputs/<area-id>/_pipeline/stages/blender.manifest.json outputs/<area-id>/_pipeline/stages/blender.manifest.json

View File

@@ -2,6 +2,11 @@
## 2026-08-04 ## 2026-08-04
- 新增 `npm run preflight:area -- --config config/areas/<area>.json`:手工编辑 OSM 后可在
QGIS / Blender / Cesium 前检查 bounds、缺失 node、building way 闭合性、building
multipolygon member/ring 与显式高度。结构错误会非零退出;成功结果原子写入
`_pipeline/stages/preflight.manifest.json`,并由 `diagnose:area` / `check:area` 检查
新鲜度。该命令不属于 `build-area` 阶段,也不调用重型工具。
- 扩展 stage manifest 到完整区域链:`intermediates``reimport``blender` - 扩展 stage manifest 到完整区域链:`intermediates``reimport``blender`
`preview` 现在和既有 `cesium` / `compress` 一样,在成功后写 `preview` 现在和既有 `cesium` / `compress` 一样,在成功后写
`_pipeline/stages/<stage>.manifest.json`。前段 manifest 记录 config / OSM / `_pipeline/stages/<stage>.manifest.json`。前段 manifest 记录 config / OSM /

View File

@@ -9,7 +9,9 @@
"build:qgis": "node scripts/build-osm2streets-qgis.js", "build:qgis": "node scripts/build-osm2streets-qgis.js",
"check:area": "node scripts/check-area.js", "check:area": "node scripts/check-area.js",
"compress:glb": "node scripts/compress-glb.js", "compress:glb": "node scripts/compress-glb.js",
"diagnose:area": "node scripts/diagnose-area.js" "diagnose:area": "node scripts/diagnose-area.js",
"preflight:area": "node scripts/preflight-area.js",
"test:preflight": "node scripts/test-area-preflight.js"
}, },
"dependencies": { "dependencies": {
"osm2streets-js-node": "0.1.4" "osm2streets-js-node": "0.1.4"

View File

@@ -67,6 +67,7 @@ function parseOsm(xml) {
const nodePattern = /<node\b([^>]*?)\/>|<node\b([^>]*?)>([\s\S]*?)<\/node>/g; const nodePattern = /<node\b([^>]*?)\/>|<node\b([^>]*?)>([\s\S]*?)<\/node>/g;
for (const match of xml.matchAll(nodePattern)) { for (const match of xml.matchAll(nodePattern)) {
const attrs = xmlAttrs(match[1] || match[2] || ""); const attrs = xmlAttrs(match[1] || match[2] || "");
if (attrs.action === "delete") continue;
if (attrs.id) nodeIds.add(attrs.id); if (attrs.id) nodeIds.add(attrs.id);
nodeStats.total += 1; nodeStats.total += 1;
const tags = parseTags(match[3] || ""); const tags = parseTags(match[3] || "");
@@ -80,6 +81,8 @@ function parseOsm(xml) {
buildingsWithHeight: 0, buildingsWithHeight: 0,
buildingsWithLevels: 0, buildingsWithLevels: 0,
buildingsWithBadHeight: 0, buildingsWithBadHeight: 0,
buildingsWithBadLevels: 0,
buildingGeometryIssues: [],
missingNodeRefs: 0, missingNodeRefs: 0,
grass: 0, grass: 0,
scrub: 0, scrub: 0,
@@ -87,6 +90,7 @@ function parseOsm(xml) {
}; };
for (const match of xml.matchAll(/<way\b([^>]*)>([\s\S]*?)<\/way>/g)) { for (const match of xml.matchAll(/<way\b([^>]*)>([\s\S]*?)<\/way>/g)) {
const attrs = xmlAttrs(match[1]); const attrs = xmlAttrs(match[1]);
if (attrs.action === "delete") continue;
const body = match[2]; const body = match[2];
const tags = parseTags(body); const tags = parseTags(body);
const refs = []; const refs = [];
@@ -110,6 +114,14 @@ function parseOsm(xml) {
if (isExplicitHeight(tags)) wayStats.buildingsWithHeight += 1; if (isExplicitHeight(tags)) wayStats.buildingsWithHeight += 1;
if (tags["building:levels"]) wayStats.buildingsWithLevels += 1; if (tags["building:levels"]) wayStats.buildingsWithLevels += 1;
if (tags.height && !parseHeightMeters(tags.height)) wayStats.buildingsWithBadHeight += 1; if (tags.height && !parseHeightMeters(tags.height)) wayStats.buildingsWithBadHeight += 1;
if (tags["building:levels"] && !parseBuildingLevels(tags["building:levels"])) {
wayStats.buildingsWithBadLevels += 1;
}
if (refs.length < 4) {
wayStats.buildingGeometryIssues.push({ id: way.id, issue: "has fewer than 4 node refs" });
} else if (!way.closed) {
wayStats.buildingGeometryIssues.push({ id: way.id, issue: "is not closed" });
}
} }
if (tags.landuse === "grass") wayStats.grass += 1; if (tags.landuse === "grass") wayStats.grass += 1;
if (tags.natural === "scrub") wayStats.scrub += 1; if (tags.natural === "scrub") wayStats.scrub += 1;
@@ -122,11 +134,13 @@ function parseOsm(xml) {
buildingsWithHeight: 0, buildingsWithHeight: 0,
buildingsWithLevels: 0, buildingsWithLevels: 0,
buildingsWithBadHeight: 0, buildingsWithBadHeight: 0,
buildingsWithBadLevels: 0,
healthyBuildingMultipolygons: 0, healthyBuildingMultipolygons: 0,
issues: [], issues: [],
}; };
for (const match of xml.matchAll(/<relation\b([^>]*)>([\s\S]*?)<\/relation>/g)) { for (const match of xml.matchAll(/<relation\b([^>]*)>([\s\S]*?)<\/relation>/g)) {
const attrs = xmlAttrs(match[1]); const attrs = xmlAttrs(match[1]);
if (attrs.action === "delete") continue;
const body = match[2]; const body = match[2];
const tags = parseTags(body); const tags = parseTags(body);
const members = []; const members = [];
@@ -139,6 +153,9 @@ function parseOsm(xml) {
if (isExplicitHeight(tags)) relationStats.buildingsWithHeight += 1; if (isExplicitHeight(tags)) relationStats.buildingsWithHeight += 1;
if (tags["building:levels"]) relationStats.buildingsWithLevels += 1; if (tags["building:levels"]) relationStats.buildingsWithLevels += 1;
if (tags.height && !parseHeightMeters(tags.height)) relationStats.buildingsWithBadHeight += 1; if (tags.height && !parseHeightMeters(tags.height)) relationStats.buildingsWithBadHeight += 1;
if (tags["building:levels"] && !parseBuildingLevels(tags["building:levels"])) {
relationStats.buildingsWithBadLevels += 1;
}
const health = buildingRelationHealth(attrs.id || "", members, ways); const health = buildingRelationHealth(attrs.id || "", members, ways);
if (health.ok) relationStats.healthyBuildingMultipolygons += 1; if (health.ok) relationStats.healthyBuildingMultipolygons += 1;
@@ -163,7 +180,10 @@ function parseBounds(xml) {
maxLon: Number(attrs.maxlon), maxLon: Number(attrs.maxlon),
maxLat: Number(attrs.maxlat), maxLat: Number(attrs.maxlat),
}; };
return Object.values(bounds).every(Number.isFinite) ? bounds : null; return Object.values(bounds).every(Number.isFinite) &&
bounds.minLon < bounds.maxLon && bounds.minLat < bounds.maxLat
? bounds
: null;
} }
function isExplicitHeight(tags) { function isExplicitHeight(tags) {
@@ -177,6 +197,48 @@ function parseHeightMeters(value) {
return Number.isFinite(height) && height > 0 ? height : null; return Number.isFinite(height) && height > 0 ? height : null;
} }
function parseBuildingLevels(value) {
const levels = Number(String(value).trim());
return Number.isFinite(levels) && levels > 0;
}
function analyzeOsmPreflight(osm) {
const errors = [];
const warnings = [];
if (!osm.bounds) errors.push("OSM has no valid <bounds>.");
if (osm.ways.missingNodeRefs) {
errors.push(`OSM ways reference ${osm.ways.missingNodeRefs} missing node(s).`);
}
for (const issue of osm.ways.buildingGeometryIssues) {
errors.push(`Building way ${issue.id}: ${issue.issue}.`);
}
if (osm.ways.buildingsWithBadHeight || osm.relations.buildingsWithBadHeight) {
errors.push("Some building height tags could not be parsed as positive meters.");
}
for (const issue of osm.relations.issues) {
errors.push(`Building relation ${issue.id}: ${issue.issues.join("; ")}.`);
}
const badLevels = osm.ways.buildingsWithBadLevels + osm.relations.buildingsWithBadLevels;
if (badLevels) {
warnings.push(`${badLevels} building:levels tag(s) could not be parsed as positive numbers.`);
}
return {
errors: uniqueLines(errors),
warnings: uniqueLines(warnings),
summary: {
bounds: osm.bounds,
nodes: osm.nodes.total,
ways: osm.ways.total,
relations: osm.relations.total,
buildingWays: osm.ways.buildings,
buildingMultipolygons: osm.relations.buildingMultipolygons,
missingNodeRefs: osm.ways.missingNodeRefs,
buildingWayIssues: osm.ways.buildingGeometryIssues.length,
buildingRelationIssues: osm.relations.issues.length,
},
};
}
function buildingRelationHealth(id, members, ways) { function buildingRelationHealth(id, members, ways) {
const issues = []; const issues = [];
const outerMembers = members.filter((member) => member.type === "way" && member.role === "outer"); const outerMembers = members.filter((member) => member.type === "way" && member.role === "outer");
@@ -307,6 +369,15 @@ function stageManifestStatus(area, configPath = null) {
const hasReimportManifest = fs.existsSync(reimportManifest); const hasReimportManifest = fs.existsSync(reimportManifest);
const derivedConfig = path.join(area.outputs.pipelineDir, "osm2streets-qgis.config.json"); const derivedConfig = path.join(area.outputs.pipelineDir, "osm2streets-qgis.config.json");
const stages = [ const stages = [
{
stage: "preflight",
expected: fs.existsSync(stageManifestPath(area, "preflight")),
inputs: {
...(configPath ? { config: configPath } : {}),
osm: area.input,
},
outputs: {},
},
{ {
stage: "intermediates", stage: "intermediates",
expected: ( expected: (
@@ -763,6 +834,7 @@ function formatBytes(bytes) {
module.exports = { module.exports = {
BUDGETS, BUDGETS,
analyzeOsmPreflight,
analyzeArea, analyzeArea,
artifactStatus, artifactStatus,
classifyAreaQuality, classifyAreaQuality,

99
scripts/preflight-area.js Normal file
View File

@@ -0,0 +1,99 @@
#!/usr/bin/env node
"use strict";
const fs = require("fs");
const path = require("path");
const { readAreaConfig } = require("./lib/area-config");
const {
analyzeOsmPreflight,
defaultConfigPath,
parseOsm,
} = require("./lib/area-diagnostics");
const { fileRecord, writeStageManifest } = require("./lib/stage-manifest");
const repoRoot = path.resolve(__dirname, "..");
function parseArgs(argv) {
const out = {};
for (let i = 0; i < argv.length; i += 1) {
const arg = argv[i];
if (!arg.startsWith("--")) continue;
const key = arg.slice(2).replace(/-([a-z])/g, (_, c) => c.toUpperCase());
const next = argv[i + 1];
if (!next || next.startsWith("--")) {
out[key] = "true";
} else {
out[key] = next;
i += 1;
}
}
return out;
}
function main() {
const startedAt = new Date().toISOString();
const args = parseArgs(process.argv.slice(2));
const configPath = path.resolve(args.config || defaultConfigPath(repoRoot));
const area = readAreaConfig(configPath, { repoRoot });
const osm = parseOsm(fs.readFileSync(area.input, "utf8"));
const preflight = analyzeOsmPreflight(osm);
printPreflightReport(area, configPath, preflight);
if (preflight.errors.length) {
process.exitCode = 1;
return;
}
const finishedAt = new Date().toISOString();
writeStageManifest(area, {
stage: "preflight",
status: "ok",
config: configPath,
startedAt,
finishedAt,
durationMs: Date.parse(finishedAt) - Date.parse(startedAt),
inputs: {
config: fileRecord(configPath),
osm: fileRecord(area.input),
},
outputs: {},
summary: preflight.summary,
errors: preflight.errors,
warnings: preflight.warnings,
});
}
function printPreflightReport(area, configPath, preflight) {
console.log("OSM area preflight");
console.log(`Area: ${area.id}`);
console.log(`Config: ${configPath}`);
console.log(`Input: ${area.input}`);
console.log("");
console.log(
`OSM: ${preflight.summary.nodes} nodes, ${preflight.summary.ways} ways, ` +
`${preflight.summary.relations} relations`,
);
console.log(
`Buildings: ${preflight.summary.buildingWays} way(s), ` +
`${preflight.summary.buildingMultipolygons} multipolygon relation(s)`,
);
console.log(`Bounds: ${preflight.summary.bounds ? "valid" : "invalid or missing"}`);
console.log("");
const status = preflight.errors.length ? "FAIL" : "PASS";
console.log(`${status}: ${preflight.errors.length} error(s), ${preflight.warnings.length} warning(s)`);
console.log("");
printLines("Errors", preflight.errors);
console.log("");
printLines("Warnings", preflight.warnings);
}
function printLines(label, lines) {
console.log(`${label} (${lines.length})`);
if (!lines.length) {
console.log(" none");
return;
}
for (const line of lines) console.log(` - ${line}`);
}
main();

View File

@@ -0,0 +1,34 @@
#!/usr/bin/env node
"use strict";
const assert = require("assert");
const { analyzeOsmPreflight, parseOsm } = require("./lib/area-diagnostics");
function osm({ bounds = '<bounds minlon="0" minlat="0" maxlon="1" maxlat="1"/>', nodes, ways, relations = "" }) {
return `<osm>${bounds}${nodes}${ways}${relations}</osm>`;
}
function nodes(ids) {
return ids.map((id) => `<node id="${id}" lat="0" lon="0"/>`).join("");
}
function way(id, refs, tags = '<tag k="building" v="yes"/>') {
return `<way id="${id}">${refs.map((ref) => `<nd ref="${ref}"/>`).join("")}${tags}</way>`;
}
function preflight(input) {
return analyzeOsmPreflight(parseOsm(input));
}
assert.equal(preflight(osm({ nodes: nodes([1, 2, 3]), ways: way(10, [1, 2, 3, 1]) })).errors.length, 0);
assert.match(preflight(osm({ bounds: "", nodes: nodes([1, 2, 3]), ways: way(10, [1, 2, 3, 1]) })).errors.join("\n"), /bounds/);
assert.match(preflight(osm({ bounds: '<bounds minlon="1" minlat="0" maxlon="0" maxlat="1"/>', nodes: nodes([1, 2, 3]), ways: way(10, [1, 2, 3, 1]) })).errors.join("\n"), /bounds/);
assert.match(preflight(osm({ nodes: nodes([1, 2, 3]), ways: way(10, [1, 2, 9, 1]) })).errors.join("\n"), /missing node/);
assert.match(preflight(osm({ nodes: nodes([1, 2, 3]), ways: way(10, [1, 2, 3, 2]) })).errors.join("\n"), /not closed/);
assert.match(preflight(osm({ nodes: nodes([1, 2, 3]), ways: way(10, [1, 2, 3, 1], '<tag k="building" v="yes"/><tag k="height" v="many"/>') })).errors.join("\n"), /height/);
assert.equal(preflight(osm({ nodes: nodes([1, 2, 3]), ways: '<way id="10" action="delete"><tag k="building" v="yes"/></way>' })).errors.length, 0);
const brokenRelation = '<relation id="20"><member type="way" ref="99" role="outer"/><tag k="type" v="multipolygon"/><tag k="building" v="yes"/></relation>';
assert.match(preflight(osm({ nodes: nodes([1, 2, 3]), ways: "", relations: brokenRelation })).errors.join("\n"), /Building relation 20/);
console.log("OSM preflight tests passed.");