Add area diagnostics command

This commit is contained in:
2026-08-04 09:16:14 +08:00
parent 78cb69edb6
commit d4c3baf608
15 changed files with 819 additions and 165 deletions

View File

@@ -42,13 +42,14 @@ Blender 高度与线性颜色。两侧靠 `catalog.check_layers()` 对账集合
### 区域输出路径
输出路径只在 `scripts/build-area.js:74``normalizeAreaConfig()` 推导。
低层脚本读取 `_pipeline/osm2streets-qgis.config.json`,不要重新读取
输出路径只在 `scripts/lib/area-config.js``normalizeAreaConfig()` 推导。
`scripts/build-area.js``scripts/diagnose-area.js` 都必须通过 `readAreaConfig()` 读取
区域配置。低层脚本读取 `_pipeline/osm2streets-qgis.config.json`,不要重新读取
`config/areas/*.json` 或在阶段函数里现场拼路径。
新增产物时,在 `normalizeAreaConfig``outputs` 里加一项,再按需写入
`writeDerivedConfig()``build-area.js:189`。这样 `intermediates``reimport`
`blender``cesium``preview` 仍然只通过磁盘产物耦合。
新增产物时,在 `area-config.js``outputs` 里加一项,再按需写入
`writeDerivedConfig()`。这样 `intermediates``reimport``blender``cesium`
`preview` 和读-only 诊断仍然只通过磁盘产物耦合。
### 材质声明
@@ -65,18 +66,20 @@ Cesium 导出调色也属于同一个材质声明:新场景把 `catalog.MATERI
## 可接受的重复
### 份 `parseArgs`
### 份 `parseArgs`
`parseArgs` 现在重复在个独立入口:
`parseArgs` 现在重复在个独立入口:
- `scripts/build-area.js:50`
- `scripts/build-area.js:54`
- `scripts/build-osm2streets-qgis.js:153`
- `scripts/reimport-gpkg.js:93`
- `scripts/compress-glb.js:16`
- `scripts/diagnose-area.js:17`
语义一致:`--kebab-case value``kebabCase: "value"`,无值 flag 变字符串 `"true"`
这份重复目前是可接受技术债,因为三个脚本都能独立运行。改其中一处解析语义时,不要顺手
只改一份;要么保持三份一致,要么把"抽公共模块"作为独立重构并跑 parity
这份重复目前是可接受技术债,因为这些脚本都能独立运行。改其中一处解析语义时,不要顺手
只改一份;要么保持全部入口一致,要么把"抽公共模块"作为独立重构并跑对应入口检查
### JS 与 Python 的图层颜色
@@ -132,7 +135,7 @@ Cesium 导出调色也属于同一个材质声明:新场景把 `catalog.MATERI
不要因为代码相似就抽象:
- `parseArgs` 当前保持独立入口价值
- `parseArgs` 当前保持独立入口价值
- `ROAD_LAYERS``SCENE_LAYERS` 跨语言且承载不同字段
- 每个要素模块各自调用 `clip_polygon` 是模块边界,不是可消除重复
@@ -156,6 +159,6 @@ Cesium 导出调色也属于同一个材质声明:新场景把 `catalog.MATERI
| 新增一份图层名列表 | 回到旧的四份同步,漏改静默错栈 |
| 把两套颜色表统一 | 破坏 QGIS 与 Blender 各自调过的视觉结果 |
| 低层脚本直接读 `config/areas/*.json` | 两层配置边界失效 |
| 只改一份 `parseArgs` 的语义 | 三个入口行为分裂 |
| 只改一份 `parseArgs` 的语义 | 独立入口行为分裂 |
| 把要素模块裁剪逻辑挪到调用方 | 不同要素的越界处理开始漂移 |
| 只改 `export_cesium.py` 的旧回退表,不写 `MATERIALS[*]["cesium"]` | 新 `.blend` 不会携带 Cesium 导出契约 |

View File

@@ -30,8 +30,9 @@
### 读代码复用思考指南
- [ ] 准备新增第二份或第三份图层、材质、配置字段枚举
- [ ] 修改份重复的 `parseArgs` 之一:
`build-area.js:50``build-osm2streets-qgis.js:153``reimport-gpkg.js:93`
- [ ] 修改份重复的 `parseArgs` 之一:
`build-area.js:54``build-osm2streets-qgis.js:153``reimport-gpkg.js:93`
`compress-glb.js:16``diagnose-area.js:17`
- [ ] 多个要素模块都要做同一件几何预处理,比如
`water.py:9``grass.py:9``scrub.py:8` 都先 `clip_polygon`
- [ ] 低层脚本想直接读取 `config/areas/*.json`,绕开派生配置
@@ -66,7 +67,7 @@ grep -rn "要改的值" scripts blender config
- 先看它有没有读到对应包的 index 和本目录指南
- 对任何"行为没变"的结论,要求说明是否需要 parity需要却没跑就是风险
- 对任何"可以合并重复"的建议,先判断重复是不是刻意边界:
`parseArgs` 目前是可接受技术债JS/Python 图层颜色则是刻意不同步
`parseArgs` 目前是可接受技术债JS/Python 图层颜色则是刻意不同步
- 对任何"加精度、加默认值、直接覆盖文件"的建议,回到真实代码注释验证;
`reimport-gpkg.js:152-156``reimport-gpkg.js:11-13` 都是反直觉约束

View File

@@ -4,13 +4,14 @@
---
## 三个入口脚本
## 命令入口脚本
| 脚本 | 角色 | 入口方式 |
|---|---|---|
| `scripts/build-area.js` | **主入口**。读区域配置,按阶段调度 | `npm run build` / `build:area` |
| `scripts/build-osm2streets-qgis.js` | intermediates 阶段的实现 | 由 build-area 调起;`npm run build:qgis` 可单跑 |
| `scripts/reimport-gpkg.js` | reimport 阶段的实现 | 由 build-area 调起 |
| `scripts/diagnose-area.js` | 快速区域诊断只读配置、OSM 和已有产物 | `npm run diagnose:area` |
`scripts/parity.js``scripts/glb-digest.js` 是校验工具,不属于构建链,见
[产物一致性指南](../guides/artifact-parity-guide.md)。
@@ -119,10 +120,102 @@ npm run compress:glb -- --input outputs/a/a.glb --output outputs/a/a-compressed-
---
## 区域诊断命令
### 1. Scope / Trigger
`diagnose:area` 是手工编辑 OSM、排查 building relation、高度语义、植被数量和现有
产物体量时的快速读-only 检查。它不属于构建阶段,不进入 `--stages`,也不调用 QGIS、
Blender、Cesium 或 `gltf-transform`
### 2. Signatures
```bash
npm run diagnose:area
npm run diagnose:area -- --config config/areas/<area>.json
```
底层入口:
```bash
node scripts/diagnose-area.js [--config config/areas/<area>.json]
```
### 3. Contracts
- 不传 `--config` 时默认读取 `config/areas/nantaizi-lake-innovation-valley.json`
- 区域配置必须通过 `scripts/lib/area-config.js``readAreaConfig()` 归一化,和
`build-area.js` 共用同一套输出路径、压缩文件名、默认阶段和配置字段语义。
- 命令只读取:
- 区域配置
- 配置里的 OSM XML
- 已存在的 `area.outputs.*` 产物
- 已存在的默认 GLB通过 `scripts/glb-digest.js` 导出的 `digest()`
- 输出为 text report包含 OSM bounds、节点/way/relation 数量、building way、
building multipolygon relation、显式 `height` / `building:levels`、植被数量、
产物存在性、metadata 摘要、GLB size/counts/extensions 和 warnings。
- warnings 不改变退出码;配置缺失、输入 OSM 缺失、GLB 文件损坏这类无法继续读取的错误才
非零退出。
### 4. Validation & Error Matrix
| 条件 | 结果 |
|---|---|
| 配置文件不存在 | `Config file not found: <path>`,非零 |
| `id` / `input` 缺失 | `Missing config key: <key>`,非零 |
| OSM XML 不存在 | `Input OSM XML not found: <path>`,非零 |
| OSM 缺 `<bounds>` | warning不中断 |
| building multipolygon 缺 outer / unresolved way / open ring | warning不中断 |
| baseline 产物缺失 | warning不中断 |
| metadata JSON 损坏 | warning不中断 |
| GLB 存在但不是合法 GLB | 抛出 `glb-digest` 错误,非零 |
| GLB 超过保守预算 | warning不中断 |
### 5. Good/Base/Bad Cases
- Good: 手工改完 OSM 后先跑 `npm run diagnose:area -- --config ...`,确认 building
relation healthy再跑 `--stages blender,cesium`
- Base: 只生成过部分阶段时运行诊断,缺失产物以 warning 暴露,用来判断下一步该补哪个阶段。
- Bad: 把诊断做成 `build-area` 的新 stage它是读-only 工具,不应参与构建调度或产物生成。
### 6. Tests Required
- `node --check scripts/diagnose-area.js`
- `node --check scripts/lib/area-config.js`
- `node --check scripts/build-area.js`
- `node --check scripts/glb-digest.js`
- `npm run diagnose:area -- --config config/areas/nantaizi-lake-innovation-valley.json`
- 如果抽取了配置归一化,至少跑一个轻量 `build-area` 阶段确认主入口仍能调度:
`npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json --stages preview`
### 7. Wrong vs Correct
Wrong:
```js
const config = JSON.parse(fs.readFileSync("config/areas/a.json", "utf8"));
const glb = path.join(config.outputRoot, config.id, `${config.id}.glb`);
```
Correct:
```js
const { readAreaConfig } = require("./lib/area-config");
const area = readAreaConfig(configPath, { repoRoot });
const glb = area.outputs.glb;
```
---
## CLI 参数解析
三个脚本各有一份**完全相同**`parseArgs`
`build-area.js:50``build-osm2streets-qgis.js:153``reimport-gpkg.js:93`
独立入口脚本各有一份同语义`parseArgs`
- `build-area.js:54`
- `build-osm2streets-qgis.js:153`
- `reimport-gpkg.js:93`
- `compress-glb.js:16`
- `diagnose-area.js:17`
```js
--kebab-case value { kebabCase: "value" }
@@ -136,9 +229,9 @@ npm run compress:glb -- --input outputs/a/a.glb --output outputs/a/a-compressed-
- **不做校验**。未知参数被静默收集,缺失参数由下游的 `requireText` / `Number.isFinite`
报错
> 这份重复是已知的、**当前被接受的**技术债:三个脚本要能各自独立运行,抽公共模块的
> 收益还不抵引入一层依赖。改其中一份时**不要**顺手把另外两份重构掉——那是独立的决定
> 且会扩大 diff。真要抽取三处一起改并跑 parity
> 这份重复是已知的、**当前被接受的**技术债:这些脚本要能各自独立运行,抽公共模块的
> 收益还不抵引入一层依赖。改其中一份解析语义时**不要**只改一份;要么保持全部一致
> 要么把抽公共解析器作为独立重构并跑对应入口检查
---
@@ -279,7 +372,7 @@ parity 校验依赖 stage 的 stdout 标记来判断阶段是否跑到(如 `SC
| 让 `reimport` / `preview` 能从配置文件默认开启 | 恢复动作变成常规行为 |
| 新阶段忘了 `ensureFile` 前置校验 | 单跑时报底层堆栈而非人话 |
| 改 stage 的 stdout 标记 | 静默破坏 parity 契约 |
| 顺手把`parseArgs` 合并 | 扩大 diff三个脚本的独立性是刻意的 |
| 顺手把`parseArgs` 合并 | 扩大 diff独立入口的独立性是刻意的 |
---

View File

@@ -13,6 +13,7 @@
| 改九个 osm2streets 图层(增/删/改顺序/改色) | [图层表](./layer-registry.md) ← **最容易出静默错误** |
| 调 QGIS / GDAL / Blender 子进程 | [外部工具调用](./external-tools.md) |
| 加阶段、加 CLI 参数、改配置字段 | [CLI 与阶段](./cli-and-stages.md) |
| 改区域诊断命令或共享区域配置归一化 | [CLI 与阶段](./cli-and-stages.md#区域诊断命令) |
| 改预览页生成 | [../preview/](../preview/index.md) |
| 声称"纯重构,产物不变" | [产物一致性指南](../guides/artifact-parity-guide.md) |
@@ -23,7 +24,9 @@
```
config/areas/<id>.json
build-area.js — normalizeAreaConfig() 推导全部输出路径
lib/area-config.js — normalizeAreaConfig() 推导全部输出路径
▼ build-area.js — 阶段调度
_pipeline/osm2streets-qgis.config.json (派生配置)
├─[intermediates]─▶ build-osm2streets-qgis.js
@@ -75,14 +78,16 @@ config/areas/<id>.json
| 文件 | 行数 | 职责 |
|---|---|---|
| `build-area.js` | 774 | 主入口:配置归一化、阶段调度、Cesium 预览页与车辆巡航生成 |
| `build-area.js` | 745 | 主入口:区域配置读取、阶段调度、Cesium 预览页与车辆巡航生成 |
| `diagnose-area.js` | 438 | 快速诊断OSM building relation、植被统计、现有产物和 GLB digest |
| `lib/area-config.js` | 133 | 区域配置归一化与输出路径推导,供 build / diagnose 复用 |
| `build-osm2streets-qgis.js` | 1468 | intermediatesosm2streets 解析、图层拆分、人行道转角合成、GeoPackage 与 QGIS 工程生成 |
| `reimport-gpkg.js` | 179 | reimportGeoPackage → GeoJSON 反向导出 |
| `lib/scene-layers.js` | 164 | 九个图层的单一事实源 + 四个派生函数 |
| `lib/cesium-preview.js` / `.css` | 672 / 230 | 预览页运行时,见 [../preview/](../preview/index.md) |
| `normalize-lane-arrows.py` | 182 | 合并 osm2streets 的三角网箭头(跑在 QGIS Python 里) |
| `parity.js` | 270 | 产物一致性校验驱动 |
| `glb-digest.js` | 121 | GLB 结构摘要 |
| `glb-digest.js` | 132 | GLB 结构摘要CLI 和诊断脚本共用 |
---

View File

@@ -0,0 +1 @@
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}

View File

@@ -0,0 +1 @@
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}

View File

@@ -0,0 +1,40 @@
# Add area diagnostics command
## Goal
Add a fast area diagnostics CLI for OSM structure and existing output artifact health.
## Requirements
- Add a fast diagnostics CLI for an area config.
- The command must not invoke QGIS, Blender, Cesium, or GLB compression.
- The report must scan the configured OSM XML for common manual-edit risks:
bounds presence, nodes, ways, relations, building ways, building multipolygon
relations, explicit heights, building levels, malformed relation members,
unresolved member ways, open rings, and basic vegetation counts.
- The report must inspect existing output artifacts when present:
GeoJSON directory, GeoPackage, QGIS project, `.blend`, render PNG, GLB,
metadata, Cesium preview, compressed GLB, compressed metadata, and compressed
preview.
- If a GLB exists, the report should reuse the existing GLB digest logic to
summarize size, nodes, meshes, materials, images, accessors, and extensions.
- The command should emit clear warnings for likely trouble, including missing
OSM bounds, malformed building multipolygons, missing expected artifacts, GLB
over a conservative size budget, high node count, and high texture count.
- Add an npm script and README usage.
## Acceptance Criteria
- [x] `npm run diagnose:area -- --config config/areas/nantaizi-lake-innovation-valley.json`
prints a useful text report without running heavy build stages.
- [x] The diagnostics report includes OSM structure, building relation health,
vegetation counts, output artifact status, and GLB digest summary.
- [x] Missing optional output artifacts are warnings, not hard failures.
- [x] Syntax checks pass for the changed Node scripts.
- [x] README documents the new diagnostic command.
## Notes
- Keep `prd.md` focused on requirements, constraints, and acceptance criteria.
- Lightweight tasks can remain PRD-only.
- For complex tasks, add `design.md` for technical design and `implement.md` for execution planning before `task.py start`.

View File

@@ -0,0 +1,26 @@
{
"id": "add-area-diagnostics",
"name": "add-area-diagnostics",
"title": "Add area diagnostics command",
"description": "Add a fast area diagnostics CLI for OSM structure and existing output artifact health.",
"status": "in_progress",
"dev_type": null,
"scope": null,
"package": null,
"priority": "P2",
"creator": "dingkang",
"assignee": "dingkang",
"createdAt": "2026-08-04",
"completedAt": null,
"branch": null,
"base_branch": "main",
"worktree_path": null,
"commit": null,
"pr_url": null,
"subtasks": [],
"children": [],
"parent": null,
"relatedFiles": [],
"notes": "",
"meta": {}
}

View File

@@ -63,6 +63,19 @@ npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json
npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json --stages cesium,compress
```
## 区域诊断
手工编辑 OSM 或怀疑产物变大时先跑快速诊断。它只读取区域配置、OSM XML 和已有输出
文件,不会启动 QGIS、Blender 或 Cesium 构建:
```bash
npm run diagnose:area -- --config config/areas/nantaizi-lake-innovation-valley.json
```
诊断会输出 OSM bounds、building way / multipolygon relation、显式高度、植被数量、
现有产物状态,以及 GLB 的 size / nodes / meshes / materials / images / extensions。
缺少已期望的基线产物、异常 building relation、GLB 超过保守预算等会进入 `Warnings`
## 区域配置
新区域从模板复制:

View File

@@ -1,5 +1,15 @@
# Changelog
## 2026-08-04
- 新增区域快速诊断入口:`npm run diagnose:area -- --config config/areas/<area>.json`
诊断只读取区域配置、OSM XML 和已有输出,不启动 QGIS / Blender / Cesium 构建;报告
OSM bounds、building way / multipolygon relation、显式 `height` /
`building:levels`、植被数量、产物存在性、metadata 和 GLB digest并对 malformed
building relation、缺失基线产物、GLB size / nodes / images 超预算给 warning。
同时把区域配置归一化抽到 `scripts/lib/area-config.js``build-area.js`
`diagnose-area.js` 共用同一套输出路径推导,避免诊断脚本复制路径规则。
## 2026-08-03
- 修复 OSM `type=multipolygon` building relation 不渲染的问题:`parse_osm()` 现在会把

View File

@@ -7,7 +7,8 @@
"build": "node scripts/build-area.js",
"build:area": "node scripts/build-area.js",
"build:qgis": "node scripts/build-osm2streets-qgis.js",
"compress:glb": "node scripts/compress-glb.js"
"compress:glb": "node scripts/compress-glb.js",
"diagnose:area": "node scripts/diagnose-area.js"
},
"dependencies": {
"osm2streets-js-node": "0.1.4"

View File

@@ -3,13 +3,14 @@
const fs = require("fs");
const path = require("path");
const { spawnSync } = require("child_process");
const { readAreaConfig } = require("./lib/area-config");
const repoRoot = path.resolve(__dirname, "..");
const args = parseArgs(process.argv.slice(2));
const configPath = path.resolve(
args.config || path.join(repoRoot, "config", "areas", "nantaizi-lake-innovation-valley.json"),
);
const area = normalizeAreaConfig(readJson(configPath));
const area = readAreaConfig(configPath, { repoRoot });
const requestedStages = args.stages
? splitList(args.stages)
: null;
@@ -67,129 +68,6 @@ function parseArgs(argv) {
return out;
}
function readJson(file) {
if (!fs.existsSync(file)) {
throw new Error(`Config file not found: ${file}`);
}
return JSON.parse(fs.readFileSync(file, "utf8"));
}
function normalizeAreaConfig(raw) {
const id = requireText(raw.id, "id");
const input = path.resolve(requireText(raw.input, "input"));
if (!fs.existsSync(input)) {
throw new Error(`Input OSM XML not found: ${input}`);
}
const outputRoot = path.resolve(raw.outputRoot || path.join(repoRoot, "outputs"));
const outputOverrides = raw.outputs || {};
const areaDir = path.resolve(outputOverrides.areaDir || path.join(outputRoot, id));
const fileStem = outputOverrides.fileStem || id;
const compress = normalizeCompressConfig(raw.compress);
const compressedFileStem = outputOverrides.compressedFileStem ||
`${fileStem}-compressed-webp${compress.textureSize}${compress.meshopt ? "-meshopt" : ""}`;
const outputs = {
areaDir,
geojsonDir: path.resolve(outputOverrides.geojsonDir || path.join(areaDir, "osm2streets_web_out")),
gpkg: path.resolve(outputOverrides.gpkg || path.join(areaDir, `${fileStem}.gpkg`)),
qgisProject: path.resolve(outputOverrides.qgisProject || path.join(areaDir, `${fileStem}.qgz`)),
qgisPreview: path.resolve(outputOverrides.qgisPreview || path.join(areaDir, `${fileStem}-preview.png`)),
blend: path.resolve(outputOverrides.blend || path.join(areaDir, `${fileStem}.blend`)),
render: path.resolve(outputOverrides.render || path.join(areaDir, `${fileStem}.png`)),
glb: path.resolve(outputOverrides.glb || path.join(areaDir, `${fileStem}.glb`)),
metadata: path.resolve(outputOverrides.metadata || path.join(areaDir, `${fileStem}.json`)),
cesiumPreview: path.resolve(
outputOverrides.cesiumPreview || path.join(areaDir, `${fileStem}-cesium-preview.html`),
),
compressedGlb: path.resolve(
outputOverrides.compressedGlb || path.join(areaDir, `${compressedFileStem}.glb`),
),
compressedMetadata: path.resolve(
outputOverrides.compressedMetadata || path.join(areaDir, `${compressedFileStem}.json`),
),
compressedCesiumPreview: path.resolve(
outputOverrides.compressedCesiumPreview || path.join(areaDir, `${compressedFileStem}-cesium-preview.html`),
),
vehicleRoute: path.resolve(outputOverrides.vehicleRoute || path.join(areaDir, `${fileStem}-vehicle-route.json`)),
vehicleModel: path.resolve(outputOverrides.vehicleModel || path.join(areaDir, `${fileStem}-vehicle-car.gltf`)),
pipelineDir: path.resolve(outputOverrides.pipelineDir || path.join(areaDir, "_pipeline")),
};
return {
id,
input,
outputRoot,
qgisApp: raw.qgisApp || "/Applications/QGIS.app",
blenderApp: raw.blenderApp || "/Applications/Blender.app",
stages: {
intermediates: raw.stages?.intermediates ?? raw.stages?.qgis ?? true,
blender: raw.stages?.blender ?? true,
cesium: raw.stages?.cesium ?? true,
reimport: false,
preview: false,
compress: false,
},
qgis: {
arrowScale: raw.qgis?.arrowScale ?? raw.arrowScale ?? 0.8,
arrowMergeTriangles: raw.qgis?.arrowMergeTriangles ?? raw.arrowMergeTriangles ?? true,
arrowOutlineSimplifyMeters: raw.qgis?.arrowOutlineSimplifyMeters ?? raw.arrowOutlineSimplifyMeters ?? 0.05,
intersectionCornerSourceMaxDimensionMeters: raw.qgis?.intersectionCornerSourceMaxDimensionMeters ?? raw.intersectionCornerSourceMaxDimensionMeters ?? 2.6,
clipPad: raw.qgis?.clipPad ?? raw.clipPad ?? 0.002,
canvasPad: raw.qgis?.canvasPad ?? raw.canvasPad ?? 0.001,
previewPad: raw.qgis?.previewPad ?? raw.previewPad ?? 0.0007,
canvasExtent: raw.qgis?.canvasExtent ?? raw.canvasExtent ?? null,
previewExtent: raw.qgis?.previewExtent ?? raw.previewExtent ?? null,
layerPrefix: raw.qgis?.layerPrefix ?? raw.layerPrefix ?? "osm2streets",
},
osm2streets: raw.osm2streets || {
debug_each_step: false,
dual_carriageway_experiment: false,
sidepath_zipping_experiment: false,
inferred_sidewalks: true,
osm2lanes: true,
},
blender: {
treeStyle: raw.blender?.treeStyle || "natural",
officeOverrides: raw.blender?.officeOverrides || raw.blender?.office_overrides || "",
},
compress,
outputs,
};
}
function requireText(value, key) {
if (typeof value !== "string" || value.trim() === "") {
throw new Error(`Missing config key: ${key}`);
}
return value;
}
function normalizeCompressConfig(raw) {
const value = raw || {};
return {
textureSize: numberOption(value.textureSize, 768, "compress.textureSize", 64, 4096),
quality: numberOption(value.quality, 82, "compress.quality", 1, 100),
effort: numberOption(value.effort, 80, "compress.effort", 0, 100),
meshopt: booleanOption(value.meshopt, false, "compress.meshopt"),
};
}
function numberOption(value, fallback, label, min, max) {
const number = value === undefined ? fallback : Number(value);
if (!Number.isFinite(number) || number < min || number > max) {
throw new Error(`${label} must be a finite number in [${min}, ${max}]`);
}
return number;
}
function booleanOption(value, fallback, label) {
if (value === undefined) return fallback;
if (typeof value === "boolean") return value;
if (value === "true") return true;
if (value === "false") return false;
throw new Error(`${label} must be boolean`);
}
function splitList(value) {
return String(value)
.split(",")

438
scripts/diagnose-area.js Normal file
View File

@@ -0,0 +1,438 @@
#!/usr/bin/env node
"use strict";
const fs = require("fs");
const path = require("path");
const { readAreaConfig } = require("./lib/area-config");
const { digest: glbDigest } = require("./glb-digest");
const repoRoot = path.resolve(__dirname, "..");
const DEFAULT_CONFIG = path.join(repoRoot, "config", "areas", "nantaizi-lake-innovation-valley.json");
const BUDGETS = {
glbBytes: 25 * 1024 * 1024,
glbNodes: 1000,
glbImages: 24,
};
function parseArgs(argv) {
const out = {};
for (let i = 0; i < argv.length; i += 1) {
const arg = argv[i];
if (!arg.startsWith("--")) continue;
const key = arg.slice(2).replace(/-([a-z])/g, (_, c) => c.toUpperCase());
const next = argv[i + 1];
if (!next || next.startsWith("--")) {
out[key] = "true";
} else {
out[key] = next;
i += 1;
}
}
return out;
}
function xmlAttrs(text) {
const attrs = {};
for (const match of text.matchAll(/([:\w-]+)\s*=\s*("([^"]*)"|'([^']*)')/g)) {
attrs[match[1]] = match[3] !== undefined ? match[3] : match[4];
}
return attrs;
}
function parseTags(body) {
const tags = {};
for (const match of body.matchAll(/<tag\b([^>]*)\/?>/g)) {
const attrs = xmlAttrs(match[1]);
if (attrs.k) tags[attrs.k] = attrs.v || "";
}
return tags;
}
function parseOsm(xml) {
const bounds = parseBounds(xml);
const nodeIds = new Set();
const nodeStats = { total: 0, naturalTree: 0 };
const nodePattern = /<node\b([^>]*?)\/>|<node\b([^>]*?)>([\s\S]*?)<\/node>/g;
for (const match of xml.matchAll(nodePattern)) {
const attrs = xmlAttrs(match[1] || match[2] || "");
if (attrs.id) nodeIds.add(attrs.id);
nodeStats.total += 1;
const tags = parseTags(match[3] || "");
if (tags.natural === "tree") nodeStats.naturalTree += 1;
}
const ways = new Map();
const wayStats = {
total: 0,
buildings: 0,
buildingsWithHeight: 0,
buildingsWithLevels: 0,
buildingsWithBadHeight: 0,
missingNodeRefs: 0,
grass: 0,
scrub: 0,
treeRows: 0,
};
for (const match of xml.matchAll(/<way\b([^>]*)>([\s\S]*?)<\/way>/g)) {
const attrs = xmlAttrs(match[1]);
const body = match[2];
const tags = parseTags(body);
const refs = [];
for (const ndMatch of body.matchAll(/<nd\b([^>]*)\/?>/g)) {
const nd = xmlAttrs(ndMatch[1]);
if (nd.ref) refs.push(nd.ref);
}
const missingNodeRefs = refs.filter((ref) => !nodeIds.has(ref)).length;
const way = {
id: attrs.id || "",
refs,
tags,
closed: refs.length > 1 && refs[0] === refs[refs.length - 1],
missingNodeRefs,
};
if (way.id) ways.set(way.id, way);
wayStats.total += 1;
wayStats.missingNodeRefs += missingNodeRefs;
if (tags.building) {
wayStats.buildings += 1;
if (isExplicitHeight(tags)) wayStats.buildingsWithHeight += 1;
if (tags["building:levels"]) wayStats.buildingsWithLevels += 1;
if (tags.height && !parseHeightMeters(tags.height)) wayStats.buildingsWithBadHeight += 1;
}
if (tags.landuse === "grass") wayStats.grass += 1;
if (tags.natural === "scrub") wayStats.scrub += 1;
if (tags.natural === "tree_row") wayStats.treeRows += 1;
}
const relationStats = {
total: 0,
buildingMultipolygons: 0,
buildingsWithHeight: 0,
buildingsWithLevels: 0,
buildingsWithBadHeight: 0,
healthyBuildingMultipolygons: 0,
issues: [],
};
for (const match of xml.matchAll(/<relation\b([^>]*)>([\s\S]*?)<\/relation>/g)) {
const attrs = xmlAttrs(match[1]);
const body = match[2];
const tags = parseTags(body);
const members = [];
for (const memberMatch of body.matchAll(/<member\b([^>]*)\/?>/g)) {
members.push(xmlAttrs(memberMatch[1]));
}
relationStats.total += 1;
if (tags.type !== "multipolygon" || !tags.building) continue;
relationStats.buildingMultipolygons += 1;
if (isExplicitHeight(tags)) relationStats.buildingsWithHeight += 1;
if (tags["building:levels"]) relationStats.buildingsWithLevels += 1;
if (tags.height && !parseHeightMeters(tags.height)) relationStats.buildingsWithBadHeight += 1;
const health = buildingRelationHealth(attrs.id || "", members, ways);
if (health.ok) {
relationStats.healthyBuildingMultipolygons += 1;
} else {
relationStats.issues.push(health);
}
}
return {
bounds,
nodes: nodeStats,
ways: wayStats,
relations: relationStats,
};
}
function parseBounds(xml) {
const match = xml.match(/<bounds\b([^>]*)\/?>/);
if (!match) return null;
const attrs = xmlAttrs(match[1]);
const bounds = {
minLon: Number(attrs.minlon),
minLat: Number(attrs.minlat),
maxLon: Number(attrs.maxlon),
maxLat: Number(attrs.maxlat),
};
return Object.values(bounds).every(Number.isFinite) ? bounds : null;
}
function isExplicitHeight(tags) {
return Boolean(tags.height && parseHeightMeters(tags.height));
}
function parseHeightMeters(value) {
const match = String(value).trim().match(/^(-?\d+(?:\.\d+)?)/);
if (!match) return null;
const height = Number(match[1]);
return Number.isFinite(height) && height > 0 ? height : null;
}
function buildingRelationHealth(id, members, ways) {
const issues = [];
const outerMembers = members.filter((member) => member.type === "way" && member.role === "outer");
const innerMembers = members.filter((member) => member.type === "way" && member.role === "inner");
const nonWayMembers = members.filter((member) => member.type && member.type !== "way");
if (!outerMembers.length) issues.push("no outer way members");
if (nonWayMembers.length) issues.push(`${nonWayMembers.length} non-way member(s)`);
const outerHealth = ringGroupHealth(outerMembers, ways);
const innerHealth = ringGroupHealth(innerMembers, ways);
issues.push(...outerHealth.issues.map((issue) => `outer ${issue}`));
issues.push(...innerHealth.issues.map((issue) => `inner ${issue}`));
return {
id,
ok: issues.length === 0,
outerMembers: outerMembers.length,
innerMembers: innerMembers.length,
unresolvedMembers: outerHealth.unresolved + innerHealth.unresolved,
openRings: outerHealth.open + innerHealth.open,
issues,
};
}
function ringGroupHealth(members, ways) {
if (!members.length) return { issues: [], unresolved: 0, open: 0 };
const issues = [];
let unresolved = 0;
let open = 0;
const fragments = [];
for (const member of members) {
const way = ways.get(member.ref);
if (!way) {
unresolved += 1;
continue;
}
if (way.refs.length < 4) {
issues.push(`member ${member.ref} has fewer than 4 node refs`);
continue;
}
fragments.push(way.refs);
}
if (unresolved) issues.push(`${unresolved} unresolved member way(s)`);
if (!fragments.length) return { issues, unresolved, open };
if (fragments.length === 1) {
if (!isClosedRefs(fragments[0])) {
open += 1;
issues.push(`member ${members[0].ref} is not closed`);
}
return { issues, unresolved, open };
}
const endpointDegrees = new Map();
for (const refs of fragments) {
if (isClosedRefs(refs)) continue;
open += 1;
addEndpoint(endpointDegrees, refs[0]);
addEndpoint(endpointDegrees, refs[refs.length - 1]);
}
const badEndpoints = [...endpointDegrees.values()].filter((count) => count !== 2).length;
if (badEndpoints) {
issues.push(`${open} open member fragment(s) do not stitch into closed rings`);
}
return { issues, unresolved, open: badEndpoints ? open : 0 };
}
function isClosedRefs(refs) {
return refs.length > 1 && refs[0] === refs[refs.length - 1];
}
function addEndpoint(map, ref) {
map.set(ref, (map.get(ref) || 0) + 1);
}
function artifactStatus(area) {
const entries = [
["GeoJSON dir", area.outputs.geojsonDir, true, "dir"],
["GeoPackage", area.outputs.gpkg, true, "file"],
["QGIS project", area.outputs.qgisProject, true, "file"],
["QGIS preview", area.outputs.qgisPreview, true, "file"],
["Blend scene", area.outputs.blend, true, "file"],
["Render PNG", area.outputs.render, true, "file"],
["Cesium GLB", area.outputs.glb, true, "file"],
["Cesium metadata", area.outputs.metadata, true, "file"],
["Cesium preview", area.outputs.cesiumPreview, true, "file"],
["Compressed GLB", area.outputs.compressedGlb, false, "file"],
["Compressed metadata", area.outputs.compressedMetadata, false, "file"],
["Compressed preview", area.outputs.compressedCesiumPreview, false, "file"],
];
return entries.map(([label, file, expected, type]) => {
const exists = fs.existsSync(file);
const stat = exists ? fs.statSync(file) : null;
const validType = !exists || (type === "dir" ? stat.isDirectory() : stat.isFile());
const geojsonFiles = exists && type === "dir"
? fs.readdirSync(file).filter((entry) => entry.endsWith(".geojson")).length
: null;
return {
label,
path: file,
expected,
type,
exists,
validType,
bytes: stat && stat.isFile() ? stat.size : null,
geojsonFiles,
};
});
}
function metadataSummary(file, warnings) {
if (!fs.existsSync(file)) return null;
try {
const metadata = JSON.parse(fs.readFileSync(file, "utf8"));
return {
asset: metadata.asset || null,
assets: Array.isArray(metadata.assets) ? metadata.assets.length : 0,
origin: metadata.origin || metadata.center || null,
};
} catch (error) {
warnings.push(`Cesium metadata is not valid JSON: ${error.message}`);
return null;
}
}
function collectWarnings(area, osm, artifacts, glb, metadata) {
const warnings = [];
if (!osm.bounds) warnings.push("OSM has no valid <bounds>; scene extent may be wrong.");
if (osm.ways.missingNodeRefs) {
warnings.push(`OSM ways reference ${osm.ways.missingNodeRefs} missing node(s).`);
}
if (osm.ways.buildingsWithBadHeight || osm.relations.buildingsWithBadHeight) {
warnings.push("Some building height tags could not be parsed as positive meters.");
}
for (const issue of osm.relations.issues) {
warnings.push(`Building relation ${issue.id}: ${issue.issues.join("; ")}.`);
}
for (const artifact of artifacts) {
if (artifact.expected && !artifact.exists) {
warnings.push(`Expected artifact missing: ${artifact.label} (${artifact.path}).`);
} else if (artifact.exists && !artifact.validType) {
warnings.push(`Artifact has wrong type: ${artifact.label} (${artifact.path}).`);
}
}
if (glb) {
if (glb.fileBytes > BUDGETS.glbBytes) {
warnings.push(`GLB size ${mb(glb.fileBytes)} MB exceeds budget ${mb(BUDGETS.glbBytes)} MB.`);
}
if (glb.counts.nodes > BUDGETS.glbNodes) {
warnings.push(`GLB nodes ${glb.counts.nodes} exceed budget ${BUDGETS.glbNodes}.`);
}
if (glb.counts.images > BUDGETS.glbImages) {
warnings.push(`GLB images ${glb.counts.images} exceed budget ${BUDGETS.glbImages}.`);
}
}
if (metadata && metadata.assets < 1) {
warnings.push("Cesium metadata has no assets entries.");
}
if (!area.blender.treeStyle) {
warnings.push("No Blender tree style configured.");
}
return warnings;
}
function printReport(area, configPath, osm, artifacts, glb, metadata, warnings) {
console.log("Area diagnostics");
console.log(`Area: ${area.id}`);
console.log(`Config: ${configPath}`);
console.log(`Input: ${area.input}`);
console.log(`Output: ${area.outputs.areaDir}`);
console.log("");
console.log("OSM");
console.log(` Bounds: ${osm.bounds ? formatBounds(osm.bounds) : "missing"}`);
console.log(` Nodes: ${formatNumber(osm.nodes.total)} (${formatNumber(osm.nodes.naturalTree)} natural=tree)`);
console.log(` Ways: ${formatNumber(osm.ways.total)}`);
console.log(` Relations: ${formatNumber(osm.relations.total)}`);
console.log(
` Buildings: ${formatNumber(osm.ways.buildings)} way(s), ` +
`${formatNumber(osm.relations.buildingMultipolygons)} multipolygon relation(s)`,
);
console.log(
` Building heights: ${formatNumber(osm.ways.buildingsWithHeight + osm.relations.buildingsWithHeight)} ` +
`height tag(s), ${formatNumber(osm.ways.buildingsWithLevels + osm.relations.buildingsWithLevels)} ` +
"building:levels tag(s)",
);
console.log(
` Building relations healthy: ${formatNumber(osm.relations.healthyBuildingMultipolygons)} / ` +
`${formatNumber(osm.relations.buildingMultipolygons)}`,
);
console.log(
` Vegetation ways: ${formatNumber(osm.ways.grass)} grass, ` +
`${formatNumber(osm.ways.scrub)} scrub, ${formatNumber(osm.ways.treeRows)} tree_row`,
);
if (osm.relations.issues.length) {
console.log(" Relation issues:");
for (const issue of osm.relations.issues) {
console.log(` - ${issue.id}: ${issue.issues.join("; ")}`);
}
}
console.log("");
console.log("Artifacts");
for (const artifact of artifacts) {
const state = artifact.exists && artifact.validType ? "ok" : (artifact.expected ? "missing" : "absent");
const suffix = artifact.geojsonFiles !== null
? `, ${artifact.geojsonFiles} GeoJSON file(s)`
: artifact.bytes !== null ? `, ${formatBytes(artifact.bytes)}` : "";
console.log(` ${state.padEnd(7)} ${artifact.label}: ${artifact.path}${suffix}`);
}
if (metadata) {
console.log(` metadata asset: ${metadata.asset || "missing"}, assets: ${metadata.assets}`);
}
if (glb) {
console.log("");
console.log("GLB digest");
console.log(` Size: ${mb(glb.fileBytes)} MB`);
console.log(
` Counts: ${glb.counts.nodes} nodes, ${glb.counts.meshes} meshes, ` +
`${glb.counts.materials} materials, ${glb.counts.images} images, ${glb.counts.accessors} accessors`,
);
console.log(` Extensions: ${glb.extensionsUsed.length ? glb.extensionsUsed.join(", ") : "none"}`);
}
console.log("");
console.log("Warnings");
if (!warnings.length) {
console.log(" none");
} else {
for (const warning of warnings) console.log(` - ${warning}`);
}
}
function formatBounds(bounds) {
return `${bounds.minLon},${bounds.minLat} -> ${bounds.maxLon},${bounds.maxLat}`;
}
function formatNumber(value) {
return new Intl.NumberFormat("en-US").format(value);
}
function mb(bytes) {
return Number((bytes / 1024 / 1024).toFixed(2));
}
function formatBytes(bytes) {
if (bytes >= 1024 * 1024) return `${mb(bytes)} MB`;
if (bytes >= 1024) return `${Number((bytes / 1024).toFixed(1))} KB`;
return `${bytes} B`;
}
function main() {
const args = parseArgs(process.argv.slice(2));
const configPath = path.resolve(args.config || DEFAULT_CONFIG);
const area = readAreaConfig(configPath, { repoRoot });
const osm = parseOsm(fs.readFileSync(area.input, "utf8"));
const artifacts = artifactStatus(area);
const metadataWarnings = [];
const metadata = metadataSummary(area.outputs.metadata, metadataWarnings);
const glb = fs.existsSync(area.outputs.glb) ? glbDigest(area.outputs.glb) : null;
const warnings = [
...metadataWarnings,
...collectWarnings(area, osm, artifacts, glb, metadata),
];
printReport(area, configPath, osm, artifacts, glb, metadata, warnings);
}
main();

View File

@@ -102,20 +102,31 @@ function digest(file) {
};
}
const argv = process.argv.slice(2);
const file = argv.find((arg) => !arg.startsWith("--"));
if (!file) {
function main() {
const argv = process.argv.slice(2);
const file = argv.find((arg) => !arg.startsWith("--"));
if (!file) {
console.error("usage: node scripts/glb-digest.js <file.glb> [--out digest.json]");
process.exit(1);
}
const outIndex = argv.indexOf("--out");
const result = digest(path.resolve(file));
const text = `${JSON.stringify(result, null, 2)}\n`;
if (outIndex >= 0 && argv[outIndex + 1]) {
}
const outIndex = argv.indexOf("--out");
const result = digest(path.resolve(file));
const text = `${JSON.stringify(result, null, 2)}\n`;
if (outIndex >= 0 && argv[outIndex + 1]) {
const out = path.resolve(argv[outIndex + 1]);
fs.mkdirSync(path.dirname(out), { recursive: true });
fs.writeFileSync(out, text);
console.log(`GLB digest: ${out}`);
} else {
} else {
process.stdout.write(text);
}
}
if (require.main === module) {
main();
}
module.exports = {
digest,
readGlbJson,
};

133
scripts/lib/area-config.js Normal file
View File

@@ -0,0 +1,133 @@
"use strict";
const fs = require("fs");
const path = require("path");
function readAreaConfig(file, options = {}) {
if (!fs.existsSync(file)) {
throw new Error(`Config file not found: ${file}`);
}
return normalizeAreaConfig(JSON.parse(fs.readFileSync(file, "utf8")), options);
}
function normalizeAreaConfig(raw, options = {}) {
const repoRoot = options.repoRoot || path.resolve(__dirname, "..", "..");
const id = requireText(raw.id, "id");
const input = path.resolve(requireText(raw.input, "input"));
if (!fs.existsSync(input)) {
throw new Error(`Input OSM XML not found: ${input}`);
}
const outputRoot = path.resolve(raw.outputRoot || path.join(repoRoot, "outputs"));
const outputOverrides = raw.outputs || {};
const areaDir = path.resolve(outputOverrides.areaDir || path.join(outputRoot, id));
const fileStem = outputOverrides.fileStem || id;
const compress = normalizeCompressConfig(raw.compress);
const compressedFileStem = outputOverrides.compressedFileStem ||
`${fileStem}-compressed-webp${compress.textureSize}${compress.meshopt ? "-meshopt" : ""}`;
const outputs = {
areaDir,
geojsonDir: path.resolve(outputOverrides.geojsonDir || path.join(areaDir, "osm2streets_web_out")),
gpkg: path.resolve(outputOverrides.gpkg || path.join(areaDir, `${fileStem}.gpkg`)),
qgisProject: path.resolve(outputOverrides.qgisProject || path.join(areaDir, `${fileStem}.qgz`)),
qgisPreview: path.resolve(outputOverrides.qgisPreview || path.join(areaDir, `${fileStem}-preview.png`)),
blend: path.resolve(outputOverrides.blend || path.join(areaDir, `${fileStem}.blend`)),
render: path.resolve(outputOverrides.render || path.join(areaDir, `${fileStem}.png`)),
glb: path.resolve(outputOverrides.glb || path.join(areaDir, `${fileStem}.glb`)),
metadata: path.resolve(outputOverrides.metadata || path.join(areaDir, `${fileStem}.json`)),
cesiumPreview: path.resolve(
outputOverrides.cesiumPreview || path.join(areaDir, `${fileStem}-cesium-preview.html`),
),
compressedGlb: path.resolve(
outputOverrides.compressedGlb || path.join(areaDir, `${compressedFileStem}.glb`),
),
compressedMetadata: path.resolve(
outputOverrides.compressedMetadata || path.join(areaDir, `${compressedFileStem}.json`),
),
compressedCesiumPreview: path.resolve(
outputOverrides.compressedCesiumPreview || path.join(areaDir, `${compressedFileStem}-cesium-preview.html`),
),
vehicleRoute: path.resolve(outputOverrides.vehicleRoute || path.join(areaDir, `${fileStem}-vehicle-route.json`)),
vehicleModel: path.resolve(outputOverrides.vehicleModel || path.join(areaDir, `${fileStem}-vehicle-car.gltf`)),
pipelineDir: path.resolve(outputOverrides.pipelineDir || path.join(areaDir, "_pipeline")),
};
return {
id,
input,
outputRoot,
qgisApp: raw.qgisApp || "/Applications/QGIS.app",
blenderApp: raw.blenderApp || "/Applications/Blender.app",
stages: {
intermediates: raw.stages?.intermediates ?? raw.stages?.qgis ?? true,
blender: raw.stages?.blender ?? true,
cesium: raw.stages?.cesium ?? true,
reimport: false,
preview: false,
compress: false,
},
qgis: {
arrowScale: raw.qgis?.arrowScale ?? raw.arrowScale ?? 0.8,
arrowMergeTriangles: raw.qgis?.arrowMergeTriangles ?? raw.arrowMergeTriangles ?? true,
arrowOutlineSimplifyMeters: raw.qgis?.arrowOutlineSimplifyMeters ?? raw.arrowOutlineSimplifyMeters ?? 0.05,
intersectionCornerSourceMaxDimensionMeters: raw.qgis?.intersectionCornerSourceMaxDimensionMeters ?? raw.intersectionCornerSourceMaxDimensionMeters ?? 2.6,
clipPad: raw.qgis?.clipPad ?? raw.clipPad ?? 0.002,
canvasPad: raw.qgis?.canvasPad ?? raw.canvasPad ?? 0.001,
previewPad: raw.qgis?.previewPad ?? raw.previewPad ?? 0.0007,
canvasExtent: raw.qgis?.canvasExtent ?? raw.canvasExtent ?? null,
previewExtent: raw.qgis?.previewExtent ?? raw.previewExtent ?? null,
layerPrefix: raw.qgis?.layerPrefix ?? raw.layerPrefix ?? "osm2streets",
},
osm2streets: raw.osm2streets || {
debug_each_step: false,
dual_carriageway_experiment: false,
sidepath_zipping_experiment: false,
inferred_sidewalks: true,
osm2lanes: true,
},
blender: {
treeStyle: raw.blender?.treeStyle || "natural",
officeOverrides: raw.blender?.officeOverrides || raw.blender?.office_overrides || "",
},
compress,
outputs,
};
}
function requireText(value, key) {
if (typeof value !== "string" || value.trim() === "") {
throw new Error(`Missing config key: ${key}`);
}
return value;
}
function normalizeCompressConfig(raw) {
const value = raw || {};
return {
textureSize: numberOption(value.textureSize, 768, "compress.textureSize", 64, 4096),
quality: numberOption(value.quality, 82, "compress.quality", 1, 100),
effort: numberOption(value.effort, 80, "compress.effort", 0, 100),
meshopt: booleanOption(value.meshopt, false, "compress.meshopt"),
};
}
function numberOption(value, fallback, label, min, max) {
const number = value === undefined ? fallback : Number(value);
if (!Number.isFinite(number) || number < min || number > max) {
throw new Error(`${label} must be a finite number in [${min}, ${max}]`);
}
return number;
}
function booleanOption(value, fallback, label) {
if (value === undefined) return fallback;
if (typeof value === "boolean") return value;
if (value === "true") return true;
if (value === "false") return false;
throw new Error(`${label} must be boolean`);
}
module.exports = {
normalizeAreaConfig,
readAreaConfig,
};