diff --git a/.trellis/spec/pipeline/cli-and-stages.md b/.trellis/spec/pipeline/cli-and-stages.md index c3da593..51418e3 100644 --- a/.trellis/spec/pipeline/cli-and-stages.md +++ b/.trellis/spec/pipeline/cli-and-stages.md @@ -12,6 +12,7 @@ | `scripts/build-osm2streets-qgis.js` | intermediates 阶段的实现 | 由 build-area 调起；`npm run build:qgis` 可单跑 | | `scripts/reimport-gpkg.js` | reimport 阶段的实现 | 由 build-area 调起 | | `scripts/diagnose-area.js` | 快速区域诊断；只读配置、OSM 和已有产物 | `npm run diagnose:area` | +| `scripts/check-area.js` | 区域质量门；复用诊断事实源并给出 PASS/FAIL 退出码 | `npm run check:area` | `scripts/parity.js` 和 `scripts/glb-digest.js` 是校验工具，不属于构建链，见 [产物一致性指南](../guides/artifact-parity-guide.md)。 @@ -210,6 +211,91 @@ const glb = area.outputs.glb; --- +## 区域质量门命令 + +### 1. Scope / Trigger + +`check:area` 是提交或交付某个区域前的只读质量门。它复用 `scripts/lib/area-diagnostics.js` +里的 OSM、产物、metadata、stage manifest 和 GLB digest 检查，只负责把诊断结果分类为 +failure / warning 并设置退出码。 + +它不属于构建阶段，不进入 `--stages`，也不调用 QGIS、Blender、Cesium、压缩或重建。 + +### 2. Signatures + +```bash +npm run check:area +npm run check:area -- --config config/areas/.json +``` + +底层入口： + +```bash +node scripts/check-area.js [--config config/areas/.json] +``` + +### 3. Contracts + +- 不传 `--config` 时默认读取 `config/areas/nantaizi-lake-innovation-valley.json`。 +- 区域配置必须通过 `scripts/lib/area-config.js` 的 `readAreaConfig()` 归一化。 +- OSM / artifacts / metadata / GLB / manifest 的解析和检查必须来自 + `scripts/lib/area-diagnostics.js`，不要在 `check-area.js` 里复制解析逻辑。 +- 输出为短 text report，包含 area、config、output、GLB 摘要、PASS/FAIL、failure + 计数和 warning 计数。 +- 有 failure 时 `process.exitCode = 1`；只有 warning 或全绿时 `process.exitCode = 0`。 + +### 4. Validation & Error Matrix + +| 条件 | 结果 | +|---|---| +| 配置文件不存在 / `id` 缺失 / OSM 文件不存在 | 共享诊断抛错，非零 | +| OSM `` 缺失或无效 | failure | +| building multipolygon 缺 outer / unresolved way / open ring | failure | +| building `height` 不能解析为正数米 | failure | +| Cesium GLB / metadata / preview 缺失或类型错误 | failure | +| metadata JSON 损坏 | failure | +| GLB size / nodes / images 超保守预算 | failure | +| expected stage manifest 缺失、损坏或 stale | failure | +| stage manifest warning 内容包含 budget exceeded | failure | +| QGIS preview 缺失 | warning，不阻断 | +| GeoJSON / GeoPackage / QGIS project / Blend / render 缺失 | warning，不阻断 | +| OSM way 引用缺失 node | warning | +| metadata 存在但没有 `assets[]` | warning | + +### 5. Good/Base/Bad Cases + +- Good: `diagnose:area` 用于调查完整细节，`check:area` 用于提交前给 CI/人一个明确退出码。 +- Base: 旧区域只缺 QGIS preview 时，`check:area` 仍 PASS，但报告 warning。 +- Bad: `check:area` 内部重新拼输出路径或重新解析 GLB budget；这会和诊断事实源漂移。 +- Bad: 把 `check:area` 做成 `build-area --stages check`；质量门是只读命令，不是构建阶段。 + +### 6. Tests Required + +- `node --check scripts/lib/area-diagnostics.js` +- `node --check scripts/diagnose-area.js` +- `node --check scripts/check-area.js` +- `npm run diagnose:area -- --config config/areas/nantaizi-lake-innovation-valley.json` +- `npm run check:area -- --config config/areas/nantaizi-lake-innovation-valley.json` +- 用临时配置指向不存在的输出目录，确认 `node scripts/check-area.js --config ` 非零退出。 + +### 7. Wrong vs Correct + +Wrong: + +```js +const metadata = JSON.parse(fs.readFileSync("outputs/a/a.json", "utf8")); +const glb = glbDigest("outputs/a/a.glb"); +``` + +Correct: + +```js +const result = analyzeArea(configPath, { repoRoot }); +const gate = classifyAreaQuality(result); +``` + +--- + ## Stage Manifest 契约 ### 1. Scope / Trigger @@ -359,7 +445,8 @@ writeStageManifest(area, { - `build-osm2streets-qgis.js:153` - `reimport-gpkg.js:93` - `compress-glb.js:16` -- `diagnose-area.js:17` +- `diagnose-area.js:13` +- `check-area.js:13` ```js --kebab-case value → { kebabCase: "value" } diff --git a/.trellis/spec/pipeline/index.md b/.trellis/spec/pipeline/index.md index 075415a..93b8f81 100644 --- a/.trellis/spec/pipeline/index.md +++ b/.trellis/spec/pipeline/index.md @@ -13,7 +13,7 @@ | 改九个 osm2streets 图层（增/删/改顺序/改色） | [图层表](./layer-registry.md) ← **最容易出静默错误** | | 调 QGIS / GDAL / Blender 子进程 | [外部工具调用](./external-tools.md) | | 加阶段、加 CLI 参数、改配置字段 | [CLI 与阶段](./cli-and-stages.md) | -| 改区域诊断命令或共享区域配置归一化 | [CLI 与阶段](./cli-and-stages.md#区域诊断命令) | +| 改区域诊断/质量门命令或共享区域配置归一化 | [CLI 与阶段](./cli-and-stages.md#区域诊断命令) 和 [质量门](./cli-and-stages.md#区域质量门命令) | | 改 stage manifest 写入、读取或 stale 判断 | [CLI 与阶段](./cli-and-stages.md#stage-manifest-契约) | | 改预览页生成 | [../preview/](../preview/index.md) | | 声称"纯重构，产物不变" | [产物一致性指南](../guides/artifact-parity-guide.md) | @@ -84,7 +84,9 @@ config/areas/.json | 文件 | 行数 | 职责 | |---|---|---| | `build-area.js` | 815 | 主入口：区域配置读取、阶段调度、Cesium 预览页、车辆巡航和 stage manifest 写入 | -| `diagnose-area.js` | 557 | 快速诊断：OSM building relation、植被统计、现有产物、stage manifest 和 GLB digest | +| `diagnose-area.js` | 36 | 快速诊断入口：调用共享 area diagnostics 并打印完整报告 | +| `check-area.js` | 74 | 区域质量门入口：调用共享 area diagnostics，输出 PASS/FAIL 并设置退出码 | +| `lib/area-diagnostics.js` | 660 | 共享区域诊断事实源：OSM、产物、metadata、stage manifest、GLB digest 和质量门分类 | | `lib/area-config.js` | 135 | 区域配置归一化与输出路径推导，供 build / diagnose 复用 | | `lib/stage-manifest.js` | 100 | stage manifest 路径、文件记录、GLB budget warning 和原子 JSON 写入 | | `build-osm2streets-qgis.js` | 1468 | intermediates：osm2streets 解析、图层拆分、人行道转角合成、GeoPackage 与 QGIS 工程生成 | diff --git a/.trellis/tasks/08-04-add-area-quality-gate/prd.md b/.trellis/tasks/08-04-add-area-quality-gate/prd.md new file mode 100644 index 0000000..434ac60 --- /dev/null +++ b/.trellis/tasks/08-04-add-area-quality-gate/prd.md @@ -0,0 +1,41 @@ +# Add area quality gate + +## Goal + +Add a lightweight check:area command that turns diagnostics and manifests into pass/fail quality gates. + +## Requirements + +- Add a lightweight `check:area` npm command for an area quality gate. +- The command must not run QGIS, Blender, Cesium, compression, or rebuild stages. +- The command must reuse the same diagnostics source of truth used by + `diagnose:area`; avoid duplicating OSM parsing, artifact checks, GLB budget + checks, or manifest stale checks. +- The first quality gate must fail on: + - invalid or missing OSM bounds + - malformed building multipolygon relations + - missing baseline GLB / metadata / Cesium preview artifacts + - invalid metadata JSON + - GLB size / node / image budgets exceeded + - missing, invalid, or stale expected stage manifests +- Missing QGIS preview remains a warning in this first version. +- The command must print a concise pass/fail report and use exit code `0` for + pass and `1` for fail. +- README and pipeline spec must document command usage and failure policy. + +## Acceptance Criteria + +- [x] `npm run check:area -- --config config/areas/nantaizi-lake-innovation-valley.json` + passes against current nantaizi outputs. +- [x] The check report includes pass/fail counts and any warning lines. +- [x] `check:area` exits non-zero for a deliberately impossible missing-output + config or equivalent controlled failure case. +- [x] `diagnose:area` still works after any shared diagnostics refactor. +- [x] Syntax checks pass for changed Node scripts. +- [x] README and Trellis pipeline spec document `check:area`. + +## Notes + +- Keep `prd.md` focused on requirements, constraints, and acceptance criteria. +- Lightweight tasks can remain PRD-only. +- For complex tasks, add `design.md` for technical design and `implement.md` for execution planning before `task.py start`. diff --git a/README.md b/README.md index f5930a3..f4a9f4d 100644 --- a/README.md +++ b/README.md @@ -87,6 +87,22 @@ manifest 记录阶段输入/输出文件的 bytes、mtime、sha256、耗时、GL `diagnose:area` 会读取这些 manifest；如果已有 GLB 但 manifest 缺失，或者 manifest 记录的输入/输出 sha/bytes 和当前文件不一致，会在 `Stage manifests` 和 `Warnings` 里标出来。 +## 区域质量门 + +提交或交付某个区域前，跑只读质量门： + +```bash +npm run check:area -- --config config/areas/nantaizi-lake-innovation-valley.json +``` + +`check:area` 复用 `diagnose:area` 的 OSM、产物、metadata、GLB digest 和 stage manifest +检查，但输出更短的 PASS/FAIL 报告。它不会启动 QGIS、Blender、Cesium、压缩或任何重建阶段。 + +第一版会在这些条件下退出非零：OSM bounds 缺失/无效、building multipolygon relation +异常、建筑 `height` 无法按正数米解析、Cesium GLB / metadata / preview 缺失或类型错误、 +metadata JSON 损坏、GLB size / nodes / images 超过保守预算、期望存在的 stage manifest +缺失/损坏/stale。缺 QGIS preview 目前只作为 warning，不阻断。 + ## 区域配置 新区域从模板复制： diff --git a/docs/changelog.md b/docs/changelog.md index 34e19ab..cd03d5c 100644 --- a/docs/changelog.md +++ b/docs/changelog.md @@ -2,6 +2,13 @@ ## 2026-08-04 +- 新增区域质量门入口：`npm run check:area -- --config config/areas/.json`。 + 它复用 `scripts/lib/area-diagnostics.js` 的 OSM、产物、metadata、stage manifest 和 + GLB digest 检查，只读已有文件，不调用 QGIS / Blender / Cesium / 压缩 / 重建；报告 + PASS/FAIL、failure 计数和 warning 计数，并在 failure 时退出非零。第一版阻断 + OSM bounds 缺失/无效、malformed building multipolygon、坏 height、Cesium GLB / + metadata / preview 缺失、metadata JSON 损坏、GLB budget 超限、expected manifest + missing / invalid / stale；缺 QGIS preview 保持 warning。 - 新增第一版 stage manifest 契约：`cesium` 和 `compress` 阶段成功后分别写入 `outputs//_pipeline/stages/cesium.manifest.json` 与 `compress.manifest.json`。manifest 记录输入/输出文件的 path / bytes / diff --git a/package.json b/package.json index 2e20bf5..a274cdc 100644 --- a/package.json +++ b/package.json @@ -7,6 +7,7 @@ "build": "node scripts/build-area.js", "build:area": "node scripts/build-area.js", "build:qgis": "node scripts/build-osm2streets-qgis.js", + "check:area": "node scripts/check-area.js", "compress:glb": "node scripts/compress-glb.js", "diagnose:area": "node scripts/diagnose-area.js" }, diff --git a/scripts/check-area.js b/scripts/check-area.js new file mode 100644 index 0000000..6209920 --- /dev/null +++ b/scripts/check-area.js @@ -0,0 +1,74 @@ +#!/usr/bin/env node +"use strict"; + +const path = require("path"); +const { + analyzeArea, + classifyAreaQuality, + defaultConfigPath, + formatBytes, +} = require("./lib/area-diagnostics"); + +const repoRoot = path.resolve(__dirname, ".."); + +function parseArgs(argv) { + const out = {}; + for (let i = 0; i < argv.length; i += 1) { + const arg = argv[i]; + if (!arg.startsWith("--")) continue; + const key = arg.slice(2).replace(/-([a-z])/g, (_, c) => c.toUpperCase()); + const next = argv[i + 1]; + if (!next || next.startsWith("--")) { + out[key] = "true"; + } else { + out[key] = next; + i += 1; + } + } + return out; +} + +function main() { + const args = parseArgs(process.argv.slice(2)); + const configPath = path.resolve(args.config || defaultConfigPath(repoRoot)); + const result = analyzeArea(configPath, { repoRoot }); + const gate = classifyAreaQuality(result); + + printCheckReport(result, gate); + process.exitCode = gate.failures.length ? 1 : 0; +} + +function printCheckReport(result, gate) { + console.log("Area quality gate"); + console.log(`Area: ${result.area.id}`); + console.log(`Config: ${result.configPath}`); + console.log(`Output: ${result.area.outputs.areaDir}`); + if (result.glb) { + console.log( + `GLB: ${formatBytes(result.glb.fileBytes)}, ${result.glb.counts.nodes} nodes, ` + + `${result.glb.counts.images} images`, + ); + } + console.log(""); + + const status = gate.failures.length ? "FAIL" : "PASS"; + console.log(`${status}: ${gate.failures.length} failure(s), ${gate.warnings.length} warning(s)`); + console.log(""); + + console.log(`Failures (${gate.failures.length})`); + if (!gate.failures.length) { + console.log(" none"); + } else { + for (const failure of gate.failures) console.log(` - ${failure}`); + } + console.log(""); + + console.log(`Warnings (${gate.warnings.length})`); + if (!gate.warnings.length) { + console.log(" none"); + } else { + for (const warning of gate.warnings) console.log(` - ${warning}`); + } +} + +main(); diff --git a/scripts/diagnose-area.js b/scripts/diagnose-area.js index 0ba3dab..10a05ba 100644 --- a/scripts/diagnose-area.js +++ b/scripts/diagnose-area.js @@ -1,19 +1,14 @@ #!/usr/bin/env node "use strict"; -const fs = require("fs"); const path = require("path"); -const { readAreaConfig } = require("./lib/area-config"); -const { digest: glbDigest } = require("./glb-digest"); -const { fileRecord, readStageManifest, stageManifestPath } = require("./lib/stage-manifest"); +const { + analyzeArea, + defaultConfigPath, + printDiagnosticsReport, +} = require("./lib/area-diagnostics"); const repoRoot = path.resolve(__dirname, ".."); -const DEFAULT_CONFIG = path.join(repoRoot, "config", "areas", "nantaizi-lake-innovation-valley.json"); -const BUDGETS = { - glbBytes: 25 * 1024 * 1024, - glbNodes: 1000, - glbImages: 24, -}; function parseArgs(argv) { const out = {}; @@ -32,545 +27,10 @@ function parseArgs(argv) { return out; } -function xmlAttrs(text) { - const attrs = {}; - for (const match of text.matchAll(/([:\w-]+)\s*=\s*("([^"]*)"|'([^']*)')/g)) { - attrs[match[1]] = match[3] !== undefined ? match[3] : match[4]; - } - return attrs; -} - -function parseTags(body) { - const tags = {}; - for (const match of body.matchAll(/]*)\/?>/g)) { - const attrs = xmlAttrs(match[1]); - if (attrs.k) tags[attrs.k] = attrs.v || ""; - } - return tags; -} - -function parseOsm(xml) { - const bounds = parseBounds(xml); - const nodeIds = new Set(); - const nodeStats = { total: 0, naturalTree: 0 }; - const nodePattern = /]*?)\/>|]*?)>([\s\S]*?)<\/node>/g; - for (const match of xml.matchAll(nodePattern)) { - const attrs = xmlAttrs(match[1] || match[2] || ""); - if (attrs.id) nodeIds.add(attrs.id); - nodeStats.total += 1; - const tags = parseTags(match[3] || ""); - if (tags.natural === "tree") nodeStats.naturalTree += 1; - } - - const ways = new Map(); - const wayStats = { - total: 0, - buildings: 0, - buildingsWithHeight: 0, - buildingsWithLevels: 0, - buildingsWithBadHeight: 0, - missingNodeRefs: 0, - grass: 0, - scrub: 0, - treeRows: 0, - }; - for (const match of xml.matchAll(/]*)>([\s\S]*?)<\/way>/g)) { - const attrs = xmlAttrs(match[1]); - const body = match[2]; - const tags = parseTags(body); - const refs = []; - for (const ndMatch of body.matchAll(/]*)\/?>/g)) { - const nd = xmlAttrs(ndMatch[1]); - if (nd.ref) refs.push(nd.ref); - } - const missingNodeRefs = refs.filter((ref) => !nodeIds.has(ref)).length; - const way = { - id: attrs.id || "", - refs, - tags, - closed: refs.length > 1 && refs[0] === refs[refs.length - 1], - missingNodeRefs, - }; - if (way.id) ways.set(way.id, way); - wayStats.total += 1; - wayStats.missingNodeRefs += missingNodeRefs; - if (tags.building) { - wayStats.buildings += 1; - if (isExplicitHeight(tags)) wayStats.buildingsWithHeight += 1; - if (tags["building:levels"]) wayStats.buildingsWithLevels += 1; - if (tags.height && !parseHeightMeters(tags.height)) wayStats.buildingsWithBadHeight += 1; - } - if (tags.landuse === "grass") wayStats.grass += 1; - if (tags.natural === "scrub") wayStats.scrub += 1; - if (tags.natural === "tree_row") wayStats.treeRows += 1; - } - - const relationStats = { - total: 0, - buildingMultipolygons: 0, - buildingsWithHeight: 0, - buildingsWithLevels: 0, - buildingsWithBadHeight: 0, - healthyBuildingMultipolygons: 0, - issues: [], - }; - for (const match of xml.matchAll(/]*)>([\s\S]*?)<\/relation>/g)) { - const attrs = xmlAttrs(match[1]); - const body = match[2]; - const tags = parseTags(body); - const members = []; - for (const memberMatch of body.matchAll(/]*)\/?>/g)) { - members.push(xmlAttrs(memberMatch[1])); - } - relationStats.total += 1; - if (tags.type !== "multipolygon" || !tags.building) continue; - relationStats.buildingMultipolygons += 1; - if (isExplicitHeight(tags)) relationStats.buildingsWithHeight += 1; - if (tags["building:levels"]) relationStats.buildingsWithLevels += 1; - if (tags.height && !parseHeightMeters(tags.height)) relationStats.buildingsWithBadHeight += 1; - - const health = buildingRelationHealth(attrs.id || "", members, ways); - if (health.ok) { - relationStats.healthyBuildingMultipolygons += 1; - } else { - relationStats.issues.push(health); - } - } - - return { - bounds, - nodes: nodeStats, - ways: wayStats, - relations: relationStats, - }; -} - -function parseBounds(xml) { - const match = xml.match(/]*)\/?>/); - if (!match) return null; - const attrs = xmlAttrs(match[1]); - const bounds = { - minLon: Number(attrs.minlon), - minLat: Number(attrs.minlat), - maxLon: Number(attrs.maxlon), - maxLat: Number(attrs.maxlat), - }; - return Object.values(bounds).every(Number.isFinite) ? bounds : null; -} - -function isExplicitHeight(tags) { - return Boolean(tags.height && parseHeightMeters(tags.height)); -} - -function parseHeightMeters(value) { - const match = String(value).trim().match(/^(-?\d+(?:\.\d+)?)/); - if (!match) return null; - const height = Number(match[1]); - return Number.isFinite(height) && height > 0 ? height : null; -} - -function buildingRelationHealth(id, members, ways) { - const issues = []; - const outerMembers = members.filter((member) => member.type === "way" && member.role === "outer"); - const innerMembers = members.filter((member) => member.type === "way" && member.role === "inner"); - const nonWayMembers = members.filter((member) => member.type && member.type !== "way"); - if (!outerMembers.length) issues.push("no outer way members"); - if (nonWayMembers.length) issues.push(`${nonWayMembers.length} non-way member(s)`); - - const outerHealth = ringGroupHealth(outerMembers, ways); - const innerHealth = ringGroupHealth(innerMembers, ways); - issues.push(...outerHealth.issues.map((issue) => `outer ${issue}`)); - issues.push(...innerHealth.issues.map((issue) => `inner ${issue}`)); - - return { - id, - ok: issues.length === 0, - outerMembers: outerMembers.length, - innerMembers: innerMembers.length, - unresolvedMembers: outerHealth.unresolved + innerHealth.unresolved, - openRings: outerHealth.open + innerHealth.open, - issues, - }; -} - -function ringGroupHealth(members, ways) { - if (!members.length) return { issues: [], unresolved: 0, open: 0 }; - const issues = []; - let unresolved = 0; - let open = 0; - const fragments = []; - for (const member of members) { - const way = ways.get(member.ref); - if (!way) { - unresolved += 1; - continue; - } - if (way.refs.length < 4) { - issues.push(`member ${member.ref} has fewer than 4 node refs`); - continue; - } - fragments.push(way.refs); - } - if (unresolved) issues.push(`${unresolved} unresolved member way(s)`); - if (!fragments.length) return { issues, unresolved, open }; - - if (fragments.length === 1) { - if (!isClosedRefs(fragments[0])) { - open += 1; - issues.push(`member ${members[0].ref} is not closed`); - } - return { issues, unresolved, open }; - } - - const endpointDegrees = new Map(); - for (const refs of fragments) { - if (isClosedRefs(refs)) continue; - open += 1; - addEndpoint(endpointDegrees, refs[0]); - addEndpoint(endpointDegrees, refs[refs.length - 1]); - } - const badEndpoints = [...endpointDegrees.values()].filter((count) => count !== 2).length; - if (badEndpoints) { - issues.push(`${open} open member fragment(s) do not stitch into closed rings`); - } - return { issues, unresolved, open: badEndpoints ? open : 0 }; -} - -function isClosedRefs(refs) { - return refs.length > 1 && refs[0] === refs[refs.length - 1]; -} - -function addEndpoint(map, ref) { - map.set(ref, (map.get(ref) || 0) + 1); -} - -function artifactStatus(area) { - const entries = [ - ["GeoJSON dir", area.outputs.geojsonDir, true, "dir"], - ["GeoPackage", area.outputs.gpkg, true, "file"], - ["QGIS project", area.outputs.qgisProject, true, "file"], - ["QGIS preview", area.outputs.qgisPreview, true, "file"], - ["Blend scene", area.outputs.blend, true, "file"], - ["Render PNG", area.outputs.render, true, "file"], - ["Cesium GLB", area.outputs.glb, true, "file"], - ["Cesium metadata", area.outputs.metadata, true, "file"], - ["Cesium preview", area.outputs.cesiumPreview, true, "file"], - ["Compressed GLB", area.outputs.compressedGlb, false, "file"], - ["Compressed metadata", area.outputs.compressedMetadata, false, "file"], - ["Compressed preview", area.outputs.compressedCesiumPreview, false, "file"], - ]; - return entries.map(([label, file, expected, type]) => { - const exists = fs.existsSync(file); - const stat = exists ? fs.statSync(file) : null; - const validType = !exists || (type === "dir" ? stat.isDirectory() : stat.isFile()); - const geojsonFiles = exists && type === "dir" - ? fs.readdirSync(file).filter((entry) => entry.endsWith(".geojson")).length - : null; - return { - label, - path: file, - expected, - type, - exists, - validType, - bytes: stat && stat.isFile() ? stat.size : null, - geojsonFiles, - }; - }); -} - -function metadataSummary(file, warnings) { - if (!fs.existsSync(file)) return null; - try { - const metadata = JSON.parse(fs.readFileSync(file, "utf8")); - return { - asset: metadata.asset || null, - assets: Array.isArray(metadata.assets) ? metadata.assets.length : 0, - origin: metadata.origin || metadata.center || null, - }; - } catch (error) { - warnings.push(`Cesium metadata is not valid JSON: ${error.message}`); - return null; - } -} - -function stageManifestStatus(area) { - const stages = [ - { - stage: "cesium", - expected: fs.existsSync(area.outputs.glb), - inputs: { - blend: area.outputs.blend, - }, - outputs: { - glb: area.outputs.glb, - metadata: area.outputs.metadata, - cesiumPreview: area.outputs.cesiumPreview, - }, - }, - { - stage: "compress", - expected: fs.existsSync(area.outputs.compressedGlb), - inputs: { - glb: area.outputs.glb, - metadata: area.outputs.metadata, - cesiumPreview: area.outputs.cesiumPreview, - }, - outputs: { - compressedGlb: area.outputs.compressedGlb, - compressedMetadata: area.outputs.compressedMetadata, - compressedCesiumPreview: area.outputs.compressedCesiumPreview, - }, - }, - ]; - return stages.map((entry) => { - const file = stageManifestPath(area, entry.stage); - try { - const manifest = readStageManifest(area, entry.stage); - if (!manifest) { - return { - stage: entry.stage, - path: file, - expected: entry.expected, - exists: false, - valid: false, - fresh: false, - manifestWarnings: [], - issues: entry.expected ? ["manifest missing"] : [], - }; - } - const issues = [ - ...manifestFileIssues(manifest.inputs || {}, entry.inputs, "input"), - ...manifestFileIssues(manifest.outputs || {}, entry.outputs, "output"), - ]; - return { - stage: entry.stage, - path: file, - expected: entry.expected, - exists: true, - valid: true, - fresh: issues.length === 0, - finishedAt: manifest.finishedAt || null, - durationMs: manifest.durationMs ?? null, - summary: manifest.summary || null, - manifestWarnings: Array.isArray(manifest.warnings) ? manifest.warnings : [], - issues, - }; - } catch (error) { - return { - stage: entry.stage, - path: file, - expected: entry.expected, - exists: fs.existsSync(file), - valid: false, - fresh: false, - manifestWarnings: [], - issues: [`manifest unreadable: ${error.message}`], - }; - } - }); -} - -function manifestFileIssues(records, expectedFiles, label) { - const issues = []; - for (const [key, file] of Object.entries(expectedFiles)) { - const recorded = records[key]; - if (!recorded) { - issues.push(`${label} ${key} not recorded`); - continue; - } - if (!fs.existsSync(file)) { - issues.push(`${label} ${key} file missing`); - continue; - } - const current = fileRecord(file); - if (recorded.bytes !== current.bytes) { - issues.push(`${label} ${key} bytes changed`); - } else if (recorded.sha256 && recorded.sha256 !== current.sha256) { - issues.push(`${label} ${key} sha256 changed`); - } - } - return issues; -} - -function collectWarnings(area, osm, artifacts, manifests, glb, metadata) { - const warnings = []; - if (!osm.bounds) warnings.push("OSM has no valid ; scene extent may be wrong."); - if (osm.ways.missingNodeRefs) { - warnings.push(`OSM ways reference ${osm.ways.missingNodeRefs} missing node(s).`); - } - if (osm.ways.buildingsWithBadHeight || osm.relations.buildingsWithBadHeight) { - warnings.push("Some building height tags could not be parsed as positive meters."); - } - for (const issue of osm.relations.issues) { - warnings.push(`Building relation ${issue.id}: ${issue.issues.join("; ")}.`); - } - for (const artifact of artifacts) { - if (artifact.expected && !artifact.exists) { - warnings.push(`Expected artifact missing: ${artifact.label} (${artifact.path}).`); - } else if (artifact.exists && !artifact.validType) { - warnings.push(`Artifact has wrong type: ${artifact.label} (${artifact.path}).`); - } - } - for (const manifest of manifests) { - if (manifest.expected && !manifest.exists) { - warnings.push(`Expected stage manifest missing: ${manifest.stage} (${manifest.path}).`); - } else if (manifest.exists && !manifest.valid) { - warnings.push(`Stage manifest invalid: ${manifest.stage} (${manifest.issues.join("; ")}).`); - } else if (manifest.exists && !manifest.fresh) { - warnings.push(`Stage manifest stale: ${manifest.stage} (${manifest.issues.join("; ")}).`); - } - for (const warning of manifest.manifestWarnings) { - warnings.push(`Stage manifest warning (${manifest.stage}): ${warning}.`); - } - } - if (glb) { - if (glb.fileBytes > BUDGETS.glbBytes) { - warnings.push(`GLB size ${mb(glb.fileBytes)} MB exceeds budget ${mb(BUDGETS.glbBytes)} MB.`); - } - if (glb.counts.nodes > BUDGETS.glbNodes) { - warnings.push(`GLB nodes ${glb.counts.nodes} exceed budget ${BUDGETS.glbNodes}.`); - } - if (glb.counts.images > BUDGETS.glbImages) { - warnings.push(`GLB images ${glb.counts.images} exceed budget ${BUDGETS.glbImages}.`); - } - } - if (metadata && metadata.assets < 1) { - warnings.push("Cesium metadata has no assets entries."); - } - if (!area.blender.treeStyle) { - warnings.push("No Blender tree style configured."); - } - return warnings; -} - -function printReport(area, configPath, osm, artifacts, manifests, glb, metadata, warnings) { - console.log("Area diagnostics"); - console.log(`Area: ${area.id}`); - console.log(`Config: ${configPath}`); - console.log(`Input: ${area.input}`); - console.log(`Output: ${area.outputs.areaDir}`); - console.log(""); - - console.log("OSM"); - console.log(` Bounds: ${osm.bounds ? formatBounds(osm.bounds) : "missing"}`); - console.log(` Nodes: ${formatNumber(osm.nodes.total)} (${formatNumber(osm.nodes.naturalTree)} natural=tree)`); - console.log(` Ways: ${formatNumber(osm.ways.total)}`); - console.log(` Relations: ${formatNumber(osm.relations.total)}`); - console.log( - ` Buildings: ${formatNumber(osm.ways.buildings)} way(s), ` + - `${formatNumber(osm.relations.buildingMultipolygons)} multipolygon relation(s)`, - ); - console.log( - ` Building heights: ${formatNumber(osm.ways.buildingsWithHeight + osm.relations.buildingsWithHeight)} ` + - `height tag(s), ${formatNumber(osm.ways.buildingsWithLevels + osm.relations.buildingsWithLevels)} ` + - "building:levels tag(s)", - ); - console.log( - ` Building relations healthy: ${formatNumber(osm.relations.healthyBuildingMultipolygons)} / ` + - `${formatNumber(osm.relations.buildingMultipolygons)}`, - ); - console.log( - ` Vegetation ways: ${formatNumber(osm.ways.grass)} grass, ` + - `${formatNumber(osm.ways.scrub)} scrub, ${formatNumber(osm.ways.treeRows)} tree_row`, - ); - if (osm.relations.issues.length) { - console.log(" Relation issues:"); - for (const issue of osm.relations.issues) { - console.log(` - ${issue.id}: ${issue.issues.join("; ")}`); - } - } - console.log(""); - - console.log("Artifacts"); - for (const artifact of artifacts) { - const state = artifact.exists && artifact.validType ? "ok" : (artifact.expected ? "missing" : "absent"); - const suffix = artifact.geojsonFiles !== null - ? `, ${artifact.geojsonFiles} GeoJSON file(s)` - : artifact.bytes !== null ? `, ${formatBytes(artifact.bytes)}` : ""; - console.log(` ${state.padEnd(7)} ${artifact.label}: ${artifact.path}${suffix}`); - } - if (metadata) { - console.log(` metadata asset: ${metadata.asset || "missing"}, assets: ${metadata.assets}`); - } - console.log(""); - - console.log("Stage manifests"); - for (const manifest of manifests) { - let state = "absent"; - if (manifest.exists && !manifest.valid) state = "invalid"; - else if (manifest.exists && !manifest.fresh) state = "stale"; - else if (manifest.exists) state = "ok"; - else if (manifest.expected) state = "missing"; - const timing = manifest.finishedAt - ? `, finished ${manifest.finishedAt}, ${manifest.durationMs ?? "?"} ms` - : ""; - const issues = manifest.issues.length ? `, ${manifest.issues.join("; ")}` : ""; - console.log(` ${state.padEnd(7)} ${manifest.stage}: ${manifest.path}${timing}${issues}`); - for (const warning of manifest.manifestWarnings) { - console.log(` warning: ${warning}`); - } - const glbSummary = manifest.summary?.glb || manifest.summary?.compressedGlb; - if (glbSummary?.counts) { - console.log( - ` GLB: ${formatBytes(glbSummary.fileBytes)}, ${glbSummary.counts.nodes} nodes, ` + - `${glbSummary.counts.meshes} meshes, ${glbSummary.counts.images} images`, - ); - } - } - if (glb) { - console.log(""); - console.log("GLB digest"); - console.log(` Size: ${mb(glb.fileBytes)} MB`); - console.log( - ` Counts: ${glb.counts.nodes} nodes, ${glb.counts.meshes} meshes, ` + - `${glb.counts.materials} materials, ${glb.counts.images} images, ${glb.counts.accessors} accessors`, - ); - console.log(` Extensions: ${glb.extensionsUsed.length ? glb.extensionsUsed.join(", ") : "none"}`); - } - console.log(""); - - console.log("Warnings"); - if (!warnings.length) { - console.log(" none"); - } else { - for (const warning of warnings) console.log(` - ${warning}`); - } -} - -function formatBounds(bounds) { - return `${bounds.minLon},${bounds.minLat} -> ${bounds.maxLon},${bounds.maxLat}`; -} - -function formatNumber(value) { - return new Intl.NumberFormat("en-US").format(value); -} - -function mb(bytes) { - return Number((bytes / 1024 / 1024).toFixed(2)); -} - -function formatBytes(bytes) { - if (bytes >= 1024 * 1024) return `${mb(bytes)} MB`; - if (bytes >= 1024) return `${Number((bytes / 1024).toFixed(1))} KB`; - return `${bytes} B`; -} - function main() { const args = parseArgs(process.argv.slice(2)); - const configPath = path.resolve(args.config || DEFAULT_CONFIG); - const area = readAreaConfig(configPath, { repoRoot }); - const osm = parseOsm(fs.readFileSync(area.input, "utf8")); - const artifacts = artifactStatus(area); - const manifests = stageManifestStatus(area); - const metadataWarnings = []; - const metadata = metadataSummary(area.outputs.metadata, metadataWarnings); - const glb = fs.existsSync(area.outputs.glb) ? glbDigest(area.outputs.glb) : null; - const warnings = [ - ...metadataWarnings, - ...collectWarnings(area, osm, artifacts, manifests, glb, metadata), - ]; - printReport(area, configPath, osm, artifacts, manifests, glb, metadata, warnings); + const configPath = path.resolve(args.config || defaultConfigPath(repoRoot)); + printDiagnosticsReport(analyzeArea(configPath, { repoRoot })); } main(); diff --git a/scripts/lib/area-diagnostics.js b/scripts/lib/area-diagnostics.js new file mode 100644 index 0000000..71023c3 --- /dev/null +++ b/scripts/lib/area-diagnostics.js @@ -0,0 +1,670 @@ +"use strict"; + +const fs = require("fs"); +const path = require("path"); +const { readAreaConfig } = require("./area-config"); +const { digest: glbDigest } = require("../glb-digest"); +const { BUDGETS, fileRecord, readStageManifest, stageManifestPath } = require("./stage-manifest"); + +function defaultConfigPath(repoRoot) { + return path.join(repoRoot, "config", "areas", "nantaizi-lake-innovation-valley.json"); +} + +function analyzeArea(configPath, options = {}) { + const repoRoot = options.repoRoot || path.resolve(__dirname, "..", ".."); + const resolvedConfig = path.resolve(configPath || defaultConfigPath(repoRoot)); + const area = readAreaConfig(resolvedConfig, { repoRoot }); + const osm = parseOsm(fs.readFileSync(area.input, "utf8")); + const artifacts = artifactStatus(area); + const manifests = stageManifestStatus(area); + const metadataWarnings = []; + const metadata = metadataSummary(area.outputs.metadata, metadataWarnings); + const glb = fs.existsSync(area.outputs.glb) ? glbDigest(area.outputs.glb) : null; + const warnings = [ + ...metadataWarnings, + ...collectWarnings(area, osm, artifacts, manifests, glb, metadata), + ]; + return { + area, + configPath: resolvedConfig, + osm, + artifacts, + manifests, + metadata, + metadataWarnings, + glb, + warnings, + }; +} + +function xmlAttrs(text) { + const attrs = {}; + for (const match of text.matchAll(/([:\w-]+)\s*=\s*("([^"]*)"|'([^']*)')/g)) { + attrs[match[1]] = match[3] !== undefined ? match[3] : match[4]; + } + return attrs; +} + +function parseTags(body) { + const tags = {}; + for (const match of body.matchAll(/]*)\/?>/g)) { + const attrs = xmlAttrs(match[1]); + if (attrs.k) tags[attrs.k] = attrs.v || ""; + } + return tags; +} + +function parseOsm(xml) { + const bounds = parseBounds(xml); + const nodeIds = new Set(); + const nodeStats = { total: 0, naturalTree: 0 }; + const nodePattern = /]*?)\/>|]*?)>([\s\S]*?)<\/node>/g; + for (const match of xml.matchAll(nodePattern)) { + const attrs = xmlAttrs(match[1] || match[2] || ""); + if (attrs.id) nodeIds.add(attrs.id); + nodeStats.total += 1; + const tags = parseTags(match[3] || ""); + if (tags.natural === "tree") nodeStats.naturalTree += 1; + } + + const ways = new Map(); + const wayStats = { + total: 0, + buildings: 0, + buildingsWithHeight: 0, + buildingsWithLevels: 0, + buildingsWithBadHeight: 0, + missingNodeRefs: 0, + grass: 0, + scrub: 0, + treeRows: 0, + }; + for (const match of xml.matchAll(/]*)>([\s\S]*?)<\/way>/g)) { + const attrs = xmlAttrs(match[1]); + const body = match[2]; + const tags = parseTags(body); + const refs = []; + for (const ndMatch of body.matchAll(/]*)\/?>/g)) { + const nd = xmlAttrs(ndMatch[1]); + if (nd.ref) refs.push(nd.ref); + } + const missingNodeRefs = refs.filter((ref) => !nodeIds.has(ref)).length; + const way = { + id: attrs.id || "", + refs, + tags, + closed: refs.length > 1 && refs[0] === refs[refs.length - 1], + missingNodeRefs, + }; + if (way.id) ways.set(way.id, way); + wayStats.total += 1; + wayStats.missingNodeRefs += missingNodeRefs; + if (tags.building) { + wayStats.buildings += 1; + if (isExplicitHeight(tags)) wayStats.buildingsWithHeight += 1; + if (tags["building:levels"]) wayStats.buildingsWithLevels += 1; + if (tags.height && !parseHeightMeters(tags.height)) wayStats.buildingsWithBadHeight += 1; + } + if (tags.landuse === "grass") wayStats.grass += 1; + if (tags.natural === "scrub") wayStats.scrub += 1; + if (tags.natural === "tree_row") wayStats.treeRows += 1; + } + + const relationStats = { + total: 0, + buildingMultipolygons: 0, + buildingsWithHeight: 0, + buildingsWithLevels: 0, + buildingsWithBadHeight: 0, + healthyBuildingMultipolygons: 0, + issues: [], + }; + for (const match of xml.matchAll(/]*)>([\s\S]*?)<\/relation>/g)) { + const attrs = xmlAttrs(match[1]); + const body = match[2]; + const tags = parseTags(body); + const members = []; + for (const memberMatch of body.matchAll(/]*)\/?>/g)) { + members.push(xmlAttrs(memberMatch[1])); + } + relationStats.total += 1; + if (tags.type !== "multipolygon" || !tags.building) continue; + relationStats.buildingMultipolygons += 1; + if (isExplicitHeight(tags)) relationStats.buildingsWithHeight += 1; + if (tags["building:levels"]) relationStats.buildingsWithLevels += 1; + if (tags.height && !parseHeightMeters(tags.height)) relationStats.buildingsWithBadHeight += 1; + + const health = buildingRelationHealth(attrs.id || "", members, ways); + if (health.ok) relationStats.healthyBuildingMultipolygons += 1; + else relationStats.issues.push(health); + } + + return { + bounds, + nodes: nodeStats, + ways: wayStats, + relations: relationStats, + }; +} + +function parseBounds(xml) { + const match = xml.match(/]*)\/?>/); + if (!match) return null; + const attrs = xmlAttrs(match[1]); + const bounds = { + minLon: Number(attrs.minlon), + minLat: Number(attrs.minlat), + maxLon: Number(attrs.maxlon), + maxLat: Number(attrs.maxlat), + }; + return Object.values(bounds).every(Number.isFinite) ? bounds : null; +} + +function isExplicitHeight(tags) { + return Boolean(tags.height && parseHeightMeters(tags.height)); +} + +function parseHeightMeters(value) { + const match = String(value).trim().match(/^(-?\d+(?:\.\d+)?)/); + if (!match) return null; + const height = Number(match[1]); + return Number.isFinite(height) && height > 0 ? height : null; +} + +function buildingRelationHealth(id, members, ways) { + const issues = []; + const outerMembers = members.filter((member) => member.type === "way" && member.role === "outer"); + const innerMembers = members.filter((member) => member.type === "way" && member.role === "inner"); + const nonWayMembers = members.filter((member) => member.type && member.type !== "way"); + if (!outerMembers.length) issues.push("no outer way members"); + if (nonWayMembers.length) issues.push(`${nonWayMembers.length} non-way member(s)`); + + const outerHealth = ringGroupHealth(outerMembers, ways); + const innerHealth = ringGroupHealth(innerMembers, ways); + issues.push(...outerHealth.issues.map((issue) => `outer ${issue}`)); + issues.push(...innerHealth.issues.map((issue) => `inner ${issue}`)); + + return { + id, + ok: issues.length === 0, + outerMembers: outerMembers.length, + innerMembers: innerMembers.length, + unresolvedMembers: outerHealth.unresolved + innerHealth.unresolved, + openRings: outerHealth.open + innerHealth.open, + issues, + }; +} + +function ringGroupHealth(members, ways) { + if (!members.length) return { issues: [], unresolved: 0, open: 0 }; + const issues = []; + let unresolved = 0; + let open = 0; + const fragments = []; + for (const member of members) { + const way = ways.get(member.ref); + if (!way) { + unresolved += 1; + continue; + } + if (way.refs.length < 4) { + issues.push(`member ${member.ref} has fewer than 4 node refs`); + continue; + } + fragments.push(way.refs); + } + if (unresolved) issues.push(`${unresolved} unresolved member way(s)`); + if (!fragments.length) return { issues, unresolved, open }; + + if (fragments.length === 1) { + if (!isClosedRefs(fragments[0])) { + open += 1; + issues.push(`member ${members[0].ref} is not closed`); + } + return { issues, unresolved, open }; + } + + const endpointDegrees = new Map(); + for (const refs of fragments) { + if (isClosedRefs(refs)) continue; + open += 1; + addEndpoint(endpointDegrees, refs[0]); + addEndpoint(endpointDegrees, refs[refs.length - 1]); + } + const badEndpoints = [...endpointDegrees.values()].filter((count) => count !== 2).length; + if (badEndpoints) { + issues.push(`${open} open member fragment(s) do not stitch into closed rings`); + } + return { issues, unresolved, open: badEndpoints ? open : 0 }; +} + +function isClosedRefs(refs) { + return refs.length > 1 && refs[0] === refs[refs.length - 1]; +} + +function addEndpoint(map, ref) { + map.set(ref, (map.get(ref) || 0) + 1); +} + +function artifactStatus(area) { + const entries = [ + ["GeoJSON dir", area.outputs.geojsonDir, true, "dir"], + ["GeoPackage", area.outputs.gpkg, true, "file"], + ["QGIS project", area.outputs.qgisProject, true, "file"], + ["QGIS preview", area.outputs.qgisPreview, true, "file"], + ["Blend scene", area.outputs.blend, true, "file"], + ["Render PNG", area.outputs.render, true, "file"], + ["Cesium GLB", area.outputs.glb, true, "file"], + ["Cesium metadata", area.outputs.metadata, true, "file"], + ["Cesium preview", area.outputs.cesiumPreview, true, "file"], + ["Compressed GLB", area.outputs.compressedGlb, false, "file"], + ["Compressed metadata", area.outputs.compressedMetadata, false, "file"], + ["Compressed preview", area.outputs.compressedCesiumPreview, false, "file"], + ]; + return entries.map(([label, file, expected, type]) => { + const exists = fs.existsSync(file); + const stat = exists ? fs.statSync(file) : null; + const validType = !exists || (type === "dir" ? stat.isDirectory() : stat.isFile()); + const geojsonFiles = exists && type === "dir" + ? fs.readdirSync(file).filter((entry) => entry.endsWith(".geojson")).length + : null; + return { + label, + path: file, + expected, + type, + exists, + validType, + bytes: stat && stat.isFile() ? stat.size : null, + geojsonFiles, + }; + }); +} + +function metadataSummary(file, warnings) { + if (!fs.existsSync(file)) return null; + try { + const metadata = JSON.parse(fs.readFileSync(file, "utf8")); + return { + asset: metadata.asset || null, + assets: Array.isArray(metadata.assets) ? metadata.assets.length : 0, + origin: metadata.origin || metadata.center || null, + }; + } catch (error) { + warnings.push(`Cesium metadata is not valid JSON: ${error.message}`); + return null; + } +} + +function stageManifestStatus(area) { + const stages = [ + { + stage: "cesium", + expected: fs.existsSync(area.outputs.glb), + inputs: { + blend: area.outputs.blend, + }, + outputs: { + glb: area.outputs.glb, + metadata: area.outputs.metadata, + cesiumPreview: area.outputs.cesiumPreview, + }, + }, + { + stage: "compress", + expected: fs.existsSync(area.outputs.compressedGlb), + inputs: { + glb: area.outputs.glb, + metadata: area.outputs.metadata, + cesiumPreview: area.outputs.cesiumPreview, + }, + outputs: { + compressedGlb: area.outputs.compressedGlb, + compressedMetadata: area.outputs.compressedMetadata, + compressedCesiumPreview: area.outputs.compressedCesiumPreview, + }, + }, + ]; + return stages.map((entry) => { + const file = stageManifestPath(area, entry.stage); + try { + const manifest = readStageManifest(area, entry.stage); + if (!manifest) { + return { + stage: entry.stage, + path: file, + expected: entry.expected, + exists: false, + valid: false, + fresh: false, + manifestWarnings: [], + issues: entry.expected ? ["manifest missing"] : [], + }; + } + const issues = [ + ...manifestFileIssues(manifest.inputs || {}, entry.inputs, "input"), + ...manifestFileIssues(manifest.outputs || {}, entry.outputs, "output"), + ]; + return { + stage: entry.stage, + path: file, + expected: entry.expected, + exists: true, + valid: true, + fresh: issues.length === 0, + finishedAt: manifest.finishedAt || null, + durationMs: manifest.durationMs ?? null, + summary: manifest.summary || null, + manifestWarnings: Array.isArray(manifest.warnings) ? manifest.warnings : [], + issues, + }; + } catch (error) { + return { + stage: entry.stage, + path: file, + expected: entry.expected, + exists: fs.existsSync(file), + valid: false, + fresh: false, + manifestWarnings: [], + issues: [`manifest unreadable: ${error.message}`], + }; + } + }); +} + +function manifestFileIssues(records, expectedFiles, label) { + const issues = []; + for (const [key, file] of Object.entries(expectedFiles)) { + const recorded = records[key]; + if (!recorded) { + issues.push(`${label} ${key} not recorded`); + continue; + } + if (!fs.existsSync(file)) { + issues.push(`${label} ${key} file missing`); + continue; + } + const current = fileRecord(file); + if (recorded.bytes !== current.bytes) { + issues.push(`${label} ${key} bytes changed`); + } else if (recorded.sha256 && recorded.sha256 !== current.sha256) { + issues.push(`${label} ${key} sha256 changed`); + } + } + return issues; +} + +function collectWarnings(area, osm, artifacts, manifests, glb, metadata) { + const warnings = []; + if (!osm.bounds) warnings.push("OSM has no valid ; scene extent may be wrong."); + if (osm.ways.missingNodeRefs) { + warnings.push(`OSM ways reference ${osm.ways.missingNodeRefs} missing node(s).`); + } + if (osm.ways.buildingsWithBadHeight || osm.relations.buildingsWithBadHeight) { + warnings.push("Some building height tags could not be parsed as positive meters."); + } + for (const issue of osm.relations.issues) { + warnings.push(`Building relation ${issue.id}: ${issue.issues.join("; ")}.`); + } + for (const artifact of artifacts) { + if (artifact.expected && !artifact.exists) { + warnings.push(`Expected artifact missing: ${artifact.label} (${artifact.path}).`); + } else if (artifact.exists && !artifact.validType) { + warnings.push(`Artifact has wrong type: ${artifact.label} (${artifact.path}).`); + } + } + for (const manifest of manifests) { + if (manifest.expected && !manifest.exists) { + warnings.push(`Expected stage manifest missing: ${manifest.stage} (${manifest.path}).`); + } else if (manifest.exists && !manifest.valid) { + warnings.push(`Stage manifest invalid: ${manifest.stage} (${manifest.issues.join("; ")}).`); + } else if (manifest.exists && !manifest.fresh) { + warnings.push(`Stage manifest stale: ${manifest.stage} (${manifest.issues.join("; ")}).`); + } + for (const warning of manifest.manifestWarnings) { + warnings.push(`Stage manifest warning (${manifest.stage}): ${warning}.`); + } + } + if (glb) { + if (glb.fileBytes > BUDGETS.glbBytes) { + warnings.push(`GLB size ${mb(glb.fileBytes)} MB exceeds budget ${mb(BUDGETS.glbBytes)} MB.`); + } + if (glb.counts.nodes > BUDGETS.glbNodes) { + warnings.push(`GLB nodes ${glb.counts.nodes} exceed budget ${BUDGETS.glbNodes}.`); + } + if (glb.counts.images > BUDGETS.glbImages) { + warnings.push(`GLB images ${glb.counts.images} exceed budget ${BUDGETS.glbImages}.`); + } + } + if (metadata && metadata.assets < 1) { + warnings.push("Cesium metadata has no assets entries."); + } + if (!area.blender.treeStyle) { + warnings.push("No Blender tree style configured."); + } + return warnings; +} + +function printDiagnosticsReport(result) { + const { area, configPath, osm, artifacts, manifests, glb, metadata, warnings } = result; + console.log("Area diagnostics"); + console.log(`Area: ${area.id}`); + console.log(`Config: ${configPath}`); + console.log(`Input: ${area.input}`); + console.log(`Output: ${area.outputs.areaDir}`); + console.log(""); + + console.log("OSM"); + console.log(` Bounds: ${osm.bounds ? formatBounds(osm.bounds) : "missing"}`); + console.log(` Nodes: ${formatNumber(osm.nodes.total)} (${formatNumber(osm.nodes.naturalTree)} natural=tree)`); + console.log(` Ways: ${formatNumber(osm.ways.total)}`); + console.log(` Relations: ${formatNumber(osm.relations.total)}`); + console.log( + ` Buildings: ${formatNumber(osm.ways.buildings)} way(s), ` + + `${formatNumber(osm.relations.buildingMultipolygons)} multipolygon relation(s)`, + ); + console.log( + ` Building heights: ${formatNumber(osm.ways.buildingsWithHeight + osm.relations.buildingsWithHeight)} ` + + `height tag(s), ${formatNumber(osm.ways.buildingsWithLevels + osm.relations.buildingsWithLevels)} ` + + "building:levels tag(s)", + ); + console.log( + ` Building relations healthy: ${formatNumber(osm.relations.healthyBuildingMultipolygons)} / ` + + `${formatNumber(osm.relations.buildingMultipolygons)}`, + ); + console.log( + ` Vegetation ways: ${formatNumber(osm.ways.grass)} grass, ` + + `${formatNumber(osm.ways.scrub)} scrub, ${formatNumber(osm.ways.treeRows)} tree_row`, + ); + if (osm.relations.issues.length) { + console.log(" Relation issues:"); + for (const issue of osm.relations.issues) { + console.log(` - ${issue.id}: ${issue.issues.join("; ")}`); + } + } + console.log(""); + + console.log("Artifacts"); + for (const artifact of artifacts) { + const state = artifact.exists && artifact.validType ? "ok" : (artifact.expected ? "missing" : "absent"); + const suffix = artifact.geojsonFiles !== null + ? `, ${artifact.geojsonFiles} GeoJSON file(s)` + : artifact.bytes !== null ? `, ${formatBytes(artifact.bytes)}` : ""; + console.log(` ${state.padEnd(7)} ${artifact.label}: ${artifact.path}${suffix}`); + } + if (metadata) { + console.log(` metadata asset: ${metadata.asset || "missing"}, assets: ${metadata.assets}`); + } + console.log(""); + + console.log("Stage manifests"); + for (const manifest of manifests) { + let state = "absent"; + if (manifest.exists && !manifest.valid) state = "invalid"; + else if (manifest.exists && !manifest.fresh) state = "stale"; + else if (manifest.exists) state = "ok"; + else if (manifest.expected) state = "missing"; + const timing = manifest.finishedAt + ? `, finished ${manifest.finishedAt}, ${manifest.durationMs ?? "?"} ms` + : ""; + const issues = manifest.issues.length ? `, ${manifest.issues.join("; ")}` : ""; + console.log(` ${state.padEnd(7)} ${manifest.stage}: ${manifest.path}${timing}${issues}`); + for (const warning of manifest.manifestWarnings) { + console.log(` warning: ${warning}`); + } + const glbSummary = manifest.summary?.glb || manifest.summary?.compressedGlb; + if (glbSummary?.counts) { + console.log( + ` GLB: ${formatBytes(glbSummary.fileBytes)}, ${glbSummary.counts.nodes} nodes, ` + + `${glbSummary.counts.meshes} meshes, ${glbSummary.counts.images} images`, + ); + } + } + if (glb) { + console.log(""); + console.log("GLB digest"); + console.log(` Size: ${mb(glb.fileBytes)} MB`); + console.log( + ` Counts: ${glb.counts.nodes} nodes, ${glb.counts.meshes} meshes, ` + + `${glb.counts.materials} materials, ${glb.counts.images} images, ${glb.counts.accessors} accessors`, + ); + console.log(` Extensions: ${glb.extensionsUsed.length ? glb.extensionsUsed.join(", ") : "none"}`); + } + console.log(""); + + console.log("Warnings"); + if (!warnings.length) { + console.log(" none"); + } else { + for (const warning of warnings) console.log(` - ${warning}`); + } +} + +function classifyAreaQuality(result) { + const failures = []; + const warnings = []; + const { osm, artifacts, manifests, glb, metadata, metadataWarnings } = result; + + if (!osm.bounds) { + failures.push("OSM has no valid ."); + } + if (osm.ways.buildingsWithBadHeight || osm.relations.buildingsWithBadHeight) { + failures.push("Some building height tags could not be parsed as positive meters."); + } + for (const issue of osm.relations.issues) { + failures.push(`Building relation ${issue.id}: ${issue.issues.join("; ")}.`); + } + if (osm.ways.missingNodeRefs) { + warnings.push(`OSM ways reference ${osm.ways.missingNodeRefs} missing node(s).`); + } + + const fatalArtifacts = new Set(["Cesium GLB", "Cesium metadata", "Cesium preview"]); + for (const artifact of artifacts) { + if (fatalArtifacts.has(artifact.label)) { + if (!artifact.exists) { + failures.push(`Required artifact missing: ${artifact.label} (${artifact.path}).`); + } else if (!artifact.validType) { + failures.push(`Required artifact has wrong type: ${artifact.label} (${artifact.path}).`); + } + continue; + } + if (artifact.expected && !artifact.exists) { + warnings.push(`Expected artifact missing: ${artifact.label} (${artifact.path}).`); + } else if (artifact.expected && artifact.exists && !artifact.validType) { + failures.push(`Artifact has wrong type: ${artifact.label} (${artifact.path}).`); + } + } + + for (const warning of metadataWarnings) { + failures.push(warning); + } + if (metadata && metadata.assets < 1) { + warnings.push("Cesium metadata has no assets entries."); + } + + for (const manifest of manifests) { + if (!manifest.expected) { + if (manifest.exists && !manifest.valid) { + warnings.push(`Stage manifest invalid: ${manifest.stage} (${manifest.issues.join("; ")}).`); + } else if (manifest.exists && !manifest.fresh) { + warnings.push(`Stage manifest stale: ${manifest.stage} (${manifest.issues.join("; ")}).`); + } + for (const warning of manifest.manifestWarnings) { + warnings.push(`Stage manifest warning (${manifest.stage}): ${warning}.`); + } + continue; + } + if (manifest.expected && !manifest.exists) { + failures.push(`Expected stage manifest missing: ${manifest.stage} (${manifest.path}).`); + } else if (manifest.exists && !manifest.valid) { + failures.push(`Stage manifest invalid: ${manifest.stage} (${manifest.issues.join("; ")}).`); + } else if (manifest.exists && !manifest.fresh) { + failures.push(`Stage manifest stale: ${manifest.stage} (${manifest.issues.join("; ")}).`); + } + for (const warning of manifest.manifestWarnings) { + if (/exceeds budget/i.test(warning)) { + failures.push(`Stage manifest warning (${manifest.stage}): ${warning}.`); + } else { + warnings.push(`Stage manifest warning (${manifest.stage}): ${warning}.`); + } + } + } + + if (glb) { + if (glb.fileBytes > BUDGETS.glbBytes) { + failures.push(`GLB size ${mb(glb.fileBytes)} MB exceeds budget ${mb(BUDGETS.glbBytes)} MB.`); + } + if (glb.counts.nodes > BUDGETS.glbNodes) { + failures.push(`GLB nodes ${glb.counts.nodes} exceed budget ${BUDGETS.glbNodes}.`); + } + if (glb.counts.images > BUDGETS.glbImages) { + failures.push(`GLB images ${glb.counts.images} exceed budget ${BUDGETS.glbImages}.`); + } + } + + if (!result.area.blender.treeStyle) { + warnings.push("No Blender tree style configured."); + } + + return { + failures: uniqueLines(failures), + warnings: uniqueLines(warnings), + }; +} + +function uniqueLines(lines) { + return [...new Set(lines)]; +} + +function formatBounds(bounds) { + return `${bounds.minLon},${bounds.minLat} -> ${bounds.maxLon},${bounds.maxLat}`; +} + +function formatNumber(value) { + return new Intl.NumberFormat("en-US").format(value); +} + +function mb(bytes) { + return Number((bytes / 1024 / 1024).toFixed(2)); +} + +function formatBytes(bytes) { + if (bytes >= 1024 * 1024) return `${mb(bytes)} MB`; + if (bytes >= 1024) return `${Number((bytes / 1024).toFixed(1))} KB`; + return `${bytes} B`; +} + +module.exports = { + BUDGETS, + analyzeArea, + artifactStatus, + classifyAreaQuality, + collectWarnings, + defaultConfigPath, + formatBytes, + mb, + parseOsm, + printDiagnosticsReport, + stageManifestStatus, +};