Add configurable area asset budgets

This commit is contained in:
2026-08-04 12:28:43 +08:00
parent f78080bbfa
commit 463cb06be4
18 changed files with 443 additions and 42 deletions

View File

@@ -54,6 +54,7 @@ cp config/examples/template.json config/areas/my-area.json
| `osm2streets` | | 见下 | 透传给 osm2streets 的选项 | | `osm2streets` | | 见下 | 透传给 osm2streets 的选项 |
| `blender` | | 见下 | Blender 侧选项 | | `blender` | | 见下 | Blender 侧选项 |
| `compress` | | 见下 | 显式 `compress` 阶段的 GLB 压缩选项 | | `compress` | | 见下 | 显式 `compress` 阶段的 GLB 压缩选项 |
| `budget` | | 见下 | 区域 GLB 性能与体量预算 |
| `outputs` | | 从 `id` 推导 | 输出路径覆盖,逃生舱 | | `outputs` | | 从 `id` 推导 | 输出路径覆盖,逃生舱 |
**路径一律绝对**`normalizeAreaConfig` 对每一项都做 `path.resolve`,相对路径会 **路径一律绝对**`normalizeAreaConfig` 对每一项都做 `path.resolve`,相对路径会
@@ -132,6 +133,23 @@ cp config/examples/template.json config/areas/my-area.json
| `effort` | `80` | WebP 编码 effort范围 `0..100` | | `effort` | `80` | WebP 编码 effort范围 `0..100` |
| `meshopt` | `false` | 是否追加 `EXT_meshopt_compression`。开启前要单独验证 Cesium 兼容性 | | `meshopt` | `false` | 是否追加 `EXT_meshopt_compression`。开启前要单独验证 Cesium 兼容性 |
### `budget`
`budget``diagnose:area``check:area` 和 Cesium / compress manifest 共用的 GLB
限制。未配置时采用全局默认;用户字段统一用 MB 或整数,归一化后内部使用 bytes / counts
| 字段 | 默认 | 说明 |
|---|---:|---|
| `glbSizeMb` | `25` | GLB 文件总大小MB |
| `nodes` | `1000` | GLB node 数量 |
| `images` | `24` | GLB image 数量 |
| `triangles` | `250000` | node 实例化后的 render triangles不是唯一 mesh 的静态 triangles |
| `embeddedImageBytesMb` | `20` | GLB 内嵌图片字节MB |
| `reason` | `""` | 任一值高于默认时必填,记录区域例外原因 |
所有数值必须为正数,`nodes` / `images` / `triangles` 必须为正整数。收紧任何默认值不需要
`reason`;放宽任一默认值而没有非空 `reason` 会在配置归一化时失败。
### `outputs`(逃生舱) ### `outputs`(逃生舱)
默认全部从 `id` 推导为 `<outputRoot>/<id>/<fileStem>.<ext>`。需要定制时逐项覆盖: 默认全部从 `id` 推导为 `<outputRoot>/<id>/<fileStem>.<ext>`。需要定制时逐项覆盖:
@@ -165,7 +183,9 @@ cp config/examples/template.json config/areas/my-area.json
3. 若要传给低层脚本,加进 `writeDerivedConfig``derivedConfig` 对象 3. 若要传给低层脚本,加进 `writeDerivedConfig``derivedConfig` 对象
4. 若是数值,在消费侧加 `Number.isFinite` + 范围校验,**在任何副作用之前** 4. 若是数值,在消费侧加 `Number.isFinite` + 范围校验,**在任何副作用之前**
5. 更新 `config/examples/template.json` 5. 更新 `config/examples/template.json`
6. 更新本文档的字段表 6. 若字段影响区域质量门,确认 `diagnose:area``check:area` 和 stage manifest 共用同一
个评估 helper不能在入口脚本各自比较阈值
7. 更新本文档的字段表
若新字段产出新文件,同时在 `outputs` 里加一行路径推导。 若新字段产出新文件,同时在 `outputs` 里加一行路径推导。

View File

@@ -323,6 +323,7 @@ node scripts/check-area.js [--config config/areas/<area>.json]
| Cesium GLB / metadata / preview 缺失或类型错误 | failure | | Cesium GLB / metadata / preview 缺失或类型错误 | failure |
| metadata JSON 损坏 | failure | | metadata JSON 损坏 | failure |
| GLB size / nodes / images 超保守预算 | failure | | GLB size / nodes / images 超保守预算 | failure |
| GLB render triangles / embedded image bytes 超保守预算 | failure |
| expected stage manifest 缺失、损坏或 stale | failure | | expected stage manifest 缺失、损坏或 stale | failure |
| stage manifest warning 内容包含 budget exceeded | failure | | stage manifest warning 内容包含 budget exceeded | failure |
| QGIS preview 缺失 | warning不阻断 | | QGIS preview 缺失 | warning不阻断 |
@@ -481,6 +482,7 @@ File records use this shape:
- `summary.glb.fileBytes` - `summary.glb.fileBytes`
- `summary.glb.counts` - `summary.glb.counts`
- `summary.glb.extensionsUsed` - `summary.glb.extensionsUsed`
- `summary.budget`effective limits、usage 和 violationswarnings 来自同一个预算评估
`cesium` 会调用 preview 生成函数,但 preview HTML / route / vehicle model 的 freshness `cesium` 会调用 preview 生成函数,但 preview HTML / route / vehicle model 的 freshness
所有权属于独立 `preview` manifest。否则单跑 `--stages preview` 会把 Cesium manifest 所有权属于独立 `preview` manifest。否则单跑 `--stages preview` 会把 Cesium manifest
@@ -512,6 +514,7 @@ File records use this shape:
- `summary.options` - `summary.options`
- `summary.compressionRatio` - `summary.compressionRatio`
- `summary.savedBytes` - `summary.savedBytes`
- `summary.budget`:压缩 GLB 的 effective limits、usage 和 violations
Manifest files are written atomically via `*.tmp` then `renameSync`. Manifest files are written atomically via `*.tmp` then `renameSync`.
@@ -524,7 +527,7 @@ Manifest files are written atomically via `*.tmp` then `renameSync`.
| Manifest output 文件缺失 | 写入时抛错;诊断时报告 stale/missing | | Manifest output 文件缺失 | 写入时抛错;诊断时报告 stale/missing |
| Manifest JSON 无法解析 | 诊断报告 invalid warning | | Manifest JSON 无法解析 | 诊断报告 invalid warning |
| Manifest 记录的 input / output sha/bytes 与当前文件不同 | 诊断报告 stale warning | | Manifest 记录的 input / output sha/bytes 与当前文件不同 | 诊断报告 stale warning |
| GLB size / nodes / images 超保守预算 | manifest `warnings[]` 记录,诊断继续独立报警 | | GLB size / nodes / images / render triangles / embedded image bytes 超保守预算 | manifest `warnings[]` 记录,诊断继续独立报警 |
| `intermediates` 后留有旧 reimport manifest | 成功后删除旧 reimport manifest | | `intermediates` 后留有旧 reimport manifest | 成功后删除旧 reimport manifest |
| `reimport` 后留有旧 intermediates manifest | 成功后删除旧 intermediates manifest | | `reimport` 后留有旧 intermediates manifest | 成功后删除旧 intermediates manifest |

View File

@@ -0,0 +1 @@
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}

View File

@@ -0,0 +1,39 @@
# Area asset budgets design
## Boundaries
`scripts/lib/area-config.js` 归一化配置的 `budget``scripts/glb-digest.js` 承担 GLB
结构统计和 source/image 摘要;`scripts/lib/stage-manifest.js` 承担默认预算、覆盖合并、
判定和 warning 文本。`scripts/lib/area-diagnostics.js` 只消费这两个共享事实源,并向
diagnose/check 报告结果。
## Data Flow
```text
config budget + global defaults
-> normalizeAreaConfig()
GLB JSON -> digest() -> usage + source/image summaries
-> evaluateGlbBudget(digest, area.budget)
-> stage manifest summary/warnings
-> diagnose report / check failures
```
预算字段面向用户使用 `glbSizeMb`,内部归一化为 bytes其余计数为整数。有效预算对象应始终
完整,避免 consumers 自己回退默认值。
## Attribution
`digest()` 将每个 node 的 mesh primitives 转为 triangles并按有限、稳定的名称规则分组
building、foliage、roads、water、fountain、other。同一 mesh 被多个 node 引用时source
summary 统计实例化后的渲染 triangles它用于帧渲染复杂度而非 GLB 静态字节。图片条目从
`bufferViews[image.bufferView].byteLength` 读取 embedded bytes并按 bytes 排序。
## Compatibility
没有 `budget` 的区域仍使用当前全局 25 MB / 1000 nodes / 24 images并增加 triangles 和
embedded image bytes 的默认阈值。旧 stage manifest 可被读取;新字段只在后续 Cesium / compress
阶段写入。现有 `check:area` 保持超预算失败。
## Rollback
移除配置归一化、预算评估和扩展摘要即可回退到固定三项预算;旧 manifest 中的附加字段可忽略。

View File

@@ -0,0 +1 @@
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}

View File

@@ -0,0 +1,24 @@
# Area asset budgets implementation plan
1. 扩展 GLB digest 的 triangles、embedded image bytes、image/source summaries并添加纯 Node
fixture 测试。
2. 在 area config 归一化 `budget`,实现全局默认、数值范围与放宽例外原因校验;更新模板。
3. 在 stage-manifest 提供单一预算评估函数,并接入 Cesium/compress manifest 写入。
4. 让 area diagnostics 与质量门复用预算评估,改进 text report。
5. 添加临时配置或 mock digest 测试,证明超标阻断、放宽无理由拒绝和收紧允许。
6. 更新文档/spec/changelog运行只针对南台子的验证。
## Validation
```bash
node --check scripts/glb-digest.js
node --check scripts/lib/area-config.js
node --check scripts/lib/stage-manifest.js
node --check scripts/lib/area-diagnostics.js
npm run test:preflight
npm run diagnose:area -- --config config/areas/nantaizi-lake-innovation-valley.json
npm run check:area -- --config config/areas/nantaizi-lake-innovation-valley.json
git diff --check
```
不跑 Blender/QGIS/Cesium除非现有 manifest 的 schema 验证无法在纯 Node 层覆盖。

View File

@@ -0,0 +1,59 @@
# Add area asset budgets
## Goal
让区域 GLB 的性能和体量约束成为可配置、可追踪、可定位的契约。手工改 OSM、资产或场景
生成逻辑后,使用者应能知道是否超出预算,以及节点、三角面和贴图主要来自哪一类资产。
## Confirmed Facts
- 原有全局保守阈值为 25 MB、1000 nodes、24 images本任务将它扩展为统一预算评估
新增 250k render triangles 与 20 MB embedded image bytes。
- `scripts/glb-digest.js` 已可无依赖读取 GLB JSON chunk且提供 node、mesh、material、
image、accessor 和 primitive vertex/index 计数。图片可通过 image 的 `bufferView` 计算嵌入
字节;三角面可由 primitive index count 计算。
- 现有南台子基线为 21.37 MB、933 nodes、117 meshes、18 images当前未超固定预算。
- GLB node/mesh 命名已经包含可用来源线索,如 `Building_*``Shapespark_*` 与道路/植被
资产名;可用于诊断“渲染实例和三角面”来源,但不能把交织二进制 buffer 的静态文件字节精确
分配给单一节点。
- `check:area` 已将超预算作为 failure新增策略必须保持这个默认阻断方向。
## Requirements
1. 区域配置新增可选 `budget` 对象,可覆盖全局默认的 GLB size、nodes、images、triangles
和 embedded image bytes 限额;未配置时保持现有默认约束,并为新增指标提供保守默认值。
2. 任何比全局默认更宽松的区域覆盖必须提供非空 `budget.reason`,使例外可审计;更严格覆盖
不需要理由。
3. 扩展共享 GLB digest提供总 triangles、embedded image bytes以及按稳定 node-name 分类的
source summary节点数、mesh instances、triangles。不虚构无法可靠归因的“每类文件字节”。
4. `diagnose:area` 显示实际值、有效预算、超标项,以及 top source / top embedded images。
5. `check:area` 复用相同预算计算,将任何超标项作为 failure不在入口脚本复制阈值或 GLB
解析逻辑。
6. Cesium/compress stage manifest 写入其对应 GLB 的 budget、usage 和 budget warnings
之后检查 stale 或回溯。
7. 更新区域配置模板、README、pipeline/config specs 和 changelog只验证南台子。
## Acceptance Criteria
- [x] `normalizeAreaConfig()` 对合法 budget 覆盖归一化,对负数、非数值或放宽默认值但缺
`reason` 的配置明确报错。
- [x] 南台子 diagnostics 显示有效预算、总 triangles、embedded image bytes、source summary
与 top images且当前默认 GLB 仍通过质量门。
- [x] 临时配置/fixture 将任一预算压低后,`check:area` 以非零退出并指出实际值和阈值。
- [x] stage manifest 包含和当前 GLB digest 对应的 budget/usage/warnings不重跑 Blender 或
Cesium 也可通过独立逻辑测试验证 payload。
- [x] 对 GLB digest 的新统计和预算分类有纯 Node 回归测试。
- [x] Node 语法检查、相关测试和 `git diff --check` 通过。
## Out Of Scope
- 不自动压缩、删减或替换资产;预算只报告和阻断。
- 不改变默认 GLB、Cesium preview 或 Blender 的视觉结果。
- 不尝试对共享/交织 GLB buffer 做不可靠的按节点文件字节归因。
- 不验证 `hanyang-block`
## Key Decisions
- 默认预算仍是阻断性质量门;区域只可通过显式配置覆盖。
- 放宽全局默认的例外必须在配置中写明原因;收紧预算不需理由。
- 归因以准确的节点实例/三角面和嵌入图片字节为准,不把静态 GLB 总字节伪分配给资产类别。

View File

@@ -0,0 +1,26 @@
{
"id": "add-area-asset-budgets",
"name": "add-area-asset-budgets",
"title": "Add area asset budgets",
"description": "Make GLB asset budgets configurable, visible in diagnostics, and enforceable by check:area.",
"status": "in_progress",
"dev_type": null,
"scope": null,
"package": null,
"priority": "P2",
"creator": "dingkang",
"assignee": "dingkang",
"createdAt": "2026-08-04",
"completedAt": null,
"branch": null,
"base_branch": "main",
"worktree_path": null,
"commit": null,
"pr_url": null,
"subtasks": [],
"children": [],
"parent": null,
"relatedFiles": [],
"notes": "",
"meta": {}
}

View File

@@ -120,9 +120,21 @@ npm run check:area -- --config config/areas/nantaizi-lake-innovation-valley.json
`check:area` 复用 `diagnose:area` 的 OSM、产物、metadata、GLB digest 和 stage manifest `check:area` 复用 `diagnose:area` 的 OSM、产物、metadata、GLB digest 和 stage manifest
检查,但输出更短的 PASS/FAIL 报告。它不会启动 QGIS、Blender、Cesium、压缩或任何重建阶段。 检查,但输出更短的 PASS/FAIL 报告。它不会启动 QGIS、Blender、Cesium、压缩或任何重建阶段。
GLB 预算默认阻断 size、nodes、images、实例化后的 render triangles 和嵌入贴图字节。
`diagnose:area` 会显示有效预算、top source 和最大贴图。区域可在配置中收紧预算;要放宽
默认值,必须写明原因:
```json
"budget": {
"nodes": 1400,
"triangles": 350000,
"reason": "Dense campus vegetation approved for this area"
}
```
第一版会在这些条件下退出非零OSM bounds 缺失/无效、building multipolygon relation 第一版会在这些条件下退出非零OSM bounds 缺失/无效、building multipolygon relation
异常、建筑 `height` 无法按正数米解析、Cesium GLB / metadata / preview 缺失或类型错误、 异常、建筑 `height` 无法按正数米解析、Cesium GLB / metadata / preview 缺失或类型错误、
metadata JSON 损坏、GLB size / nodes / images 超过保守预算、期望存在的 stage manifest metadata JSON 损坏、GLB size / nodes / images / render triangles / embedded image bytes 超过保守预算、期望存在的 stage manifest
缺失/损坏/stale。缺 QGIS preview 目前只作为 warning不阻断。 缺失/损坏/stale。缺 QGIS preview 目前只作为 warning不阻断。
## 区域配置 ## 区域配置

View File

@@ -37,5 +37,13 @@
"quality": 82, "quality": 82,
"effort": 80, "effort": 80,
"meshopt": false "meshopt": false
},
"budget": {
"glbSizeMb": 25,
"nodes": 1000,
"images": 24,
"triangles": 250000,
"embeddedImageBytesMb": 20,
"reason": ""
} }
} }

View File

@@ -2,6 +2,10 @@
## 2026-08-04 ## 2026-08-04
- GLB 预算现在由区域配置契约化:`budget.glbSizeMb``nodes``images``triangles`
`embeddedImageBytesMb` 可覆盖默认限制;任何放宽必须有 `budget.reason`。共享 digest 新增
静态/实例化 render triangles、嵌入贴图字节、节点来源和最大贴图摘要`diagnose:area`
`check:area` 与 Cesium/compress manifest 复用同一预算评估,避免阈值漂移。
- 新增 `npm run preflight:area -- --config config/areas/<area>.json`:手工编辑 OSM 后可在 - 新增 `npm run preflight:area -- --config config/areas/<area>.json`:手工编辑 OSM 后可在
QGIS / Blender / Cesium 前检查 bounds、缺失 node、building way 闭合性、building QGIS / Blender / Cesium 前检查 bounds、缺失 node、building way 闭合性、building
multipolygon member/ring 与显式高度。结构错误会非零退出;成功结果原子写入 multipolygon member/ring 与显式高度。结构错误会非零退出;成功结果原子写入

View File

@@ -11,7 +11,8 @@
"compress:glb": "node scripts/compress-glb.js", "compress:glb": "node scripts/compress-glb.js",
"diagnose:area": "node scripts/diagnose-area.js", "diagnose:area": "node scripts/diagnose-area.js",
"preflight:area": "node scripts/preflight-area.js", "preflight:area": "node scripts/preflight-area.js",
"test:preflight": "node scripts/test-area-preflight.js" "test:preflight": "node scripts/test-area-preflight.js",
"test:budgets": "node scripts/test-asset-budgets.js"
}, },
"dependencies": { "dependencies": {
"osm2streets-js-node": "0.1.4" "osm2streets-js-node": "0.1.4"

View File

@@ -13,6 +13,7 @@ const {
const { digest: glbDigest } = require("./glb-digest"); const { digest: glbDigest } = require("./glb-digest");
const { const {
fileRecord, fileRecord,
evaluateGlbBudget,
glbBudgetWarnings, glbBudgetWarnings,
glbSummary, glbSummary,
optionalFileRecord, optionalFileRecord,
@@ -334,8 +335,9 @@ function exportCesium(area) {
}, },
summary: { summary: {
glb: glbSummary(digest), glb: glbSummary(digest),
budget: evaluateGlbBudget(digest, area.budget),
}, },
warnings: glbBudgetWarnings("Cesium", digest), warnings: glbBudgetWarnings("Cesium", digest, area.budget),
}); });
} }
@@ -400,6 +402,7 @@ function compressCesiumGlb(area) {
summary: { summary: {
sourceGlb: glbSummary(sourceDigest), sourceGlb: glbSummary(sourceDigest),
compressedGlb: glbSummary(compressedDigest), compressedGlb: glbSummary(compressedDigest),
budget: evaluateGlbBudget(compressedDigest, area.budget),
options: { options: {
textureSize: area.compress.textureSize, textureSize: area.compress.textureSize,
quality: area.compress.quality, quality: area.compress.quality,
@@ -409,7 +412,7 @@ function compressCesiumGlb(area) {
compressionRatio: Number((compressedDigest.fileBytes / sourceDigest.fileBytes).toFixed(4)), compressionRatio: Number((compressedDigest.fileBytes / sourceDigest.fileBytes).toFixed(4)),
savedBytes: sourceDigest.fileBytes - compressedDigest.fileBytes, savedBytes: sourceDigest.fileBytes - compressedDigest.fileBytes,
}, },
warnings: glbBudgetWarnings("Compressed", compressedDigest), warnings: glbBudgetWarnings("Compressed", compressedDigest, area.budget),
}); });
} }

View File

@@ -59,6 +59,10 @@ function materialDigest(material) {
function digest(file) { function digest(file) {
const gltf = readGlbJson(file); const gltf = readGlbJson(file);
return digestGltf(gltf, { file: path.basename(file), fileBytes: fs.statSync(file).size });
}
function digestGltf(gltf, fileInfo = {}) {
const meshes = (gltf.meshes || []).map((mesh) => ({ const meshes = (gltf.meshes || []).map((mesh) => ({
name: mesh.name || null, name: mesh.name || null,
primitives: (mesh.primitives || []).map((primitive) => ({ primitives: (mesh.primitives || []).map((primitive) => ({
@@ -68,18 +72,37 @@ function digest(file) {
// fingerprint and stay stable regardless of buffer layout. // fingerprint and stay stable regardless of buffer layout.
count: gltf.accessors?.[primitive.attributes?.POSITION]?.count ?? null, count: gltf.accessors?.[primitive.attributes?.POSITION]?.count ?? null,
indices: gltf.accessors?.[primitive.indices]?.count ?? null, indices: gltf.accessors?.[primitive.indices]?.count ?? null,
triangles: primitiveTriangleCount(primitive, gltf.accessors || []),
})), })),
})); }));
const sourceSummary = summarizeNodeSources(gltf.nodes || [], meshes);
const images = (gltf.images || [])
.map((image) => ({
name: image.name || null,
mimeType: image.mimeType || null,
bytes: gltf.bufferViews?.[image.bufferView]?.byteLength ?? 0,
}))
.sort((a, b) => b.bytes - a.bytes || String(a.name).localeCompare(String(b.name)));
const triangles = meshes.reduce(
(total, mesh) => total + mesh.primitives.reduce((sum, primitive) => sum + primitive.triangles, 0),
0,
);
const renderTriangles = sourceSummary.reduce((total, source) => total + source.triangles, 0);
const embeddedImageBytes = images.reduce((total, image) => total + image.bytes, 0);
return { return {
file: path.basename(file), file: fileInfo.file || null,
fileBytes: fs.statSync(file).size, fileBytes: fileInfo.fileBytes ?? 0,
counts: { counts: {
nodes: (gltf.nodes || []).length, nodes: (gltf.nodes || []).length,
meshes: meshes.length, meshes: meshes.length,
materials: (gltf.materials || []).length, materials: (gltf.materials || []).length,
images: (gltf.images || []).length, images: (gltf.images || []).length,
accessors: (gltf.accessors || []).length, accessors: (gltf.accessors || []).length,
triangles,
renderTriangles,
}, },
embeddedImageBytes,
sourceSummary,
extensionsUsed: (gltf.extensionsUsed || []).slice().sort(), extensionsUsed: (gltf.extensionsUsed || []).slice().sort(),
buffers: (gltf.buffers || []).map((buffer) => buffer.byteLength), buffers: (gltf.buffers || []).map((buffer) => buffer.byteLength),
nodes: (gltf.nodes || []) nodes: (gltf.nodes || [])
@@ -96,12 +119,44 @@ function digest(file) {
materials: (gltf.materials || []) materials: (gltf.materials || [])
.map(materialDigest) .map(materialDigest)
.sort((a, b) => String(a.name).localeCompare(String(b.name))), .sort((a, b) => String(a.name).localeCompare(String(b.name))),
images: (gltf.images || []) images,
.map((image) => ({ name: image.name || null, mimeType: image.mimeType || null }))
.sort((a, b) => String(a.name).localeCompare(String(b.name))),
}; };
} }
function primitiveTriangleCount(primitive, accessors) {
if (primitive.mode !== undefined && primitive.mode !== 4) return 0;
const count = primitive.indices === undefined
? accessors[primitive.attributes?.POSITION]?.count
: accessors[primitive.indices]?.count;
return Number.isFinite(count) ? Math.floor(count / 3) : 0;
}
function summarizeNodeSources(nodes, meshes) {
const summaries = new Map();
for (const node of nodes) {
if (node.mesh === undefined || !meshes[node.mesh]) continue;
const source = nodeSource(node.name || meshes[node.mesh].name || "");
const summary = summaries.get(source) || { source, nodes: 0, meshInstances: 0, triangles: 0 };
const triangles = meshes[node.mesh].primitives.reduce((sum, primitive) => sum + primitive.triangles, 0);
summary.nodes += 1;
summary.meshInstances += 1;
summary.triangles += triangles;
summaries.set(source, summary);
}
return [...summaries.values()].sort((a, b) => b.triangles - a.triangles || a.source.localeCompare(b.source));
}
function nodeSource(name) {
const value = String(name).toLowerCase();
if (value.includes("building")) return "buildings";
if (/(shapespark|tree|bush|shrub|grass|branch|trunk|leaf)/.test(value)) return "vegetation";
if (/(road|sidewalk|lane|crosswalk|stop|intersection)/.test(value)) return "roads";
if (/(lake|water)/.test(value)) return "water";
if (value.includes("fountain")) return "fountain";
if (/(vehicle|car)/.test(value)) return "vehicles";
return "other";
}
function main() { function main() {
const argv = process.argv.slice(2); const argv = process.argv.slice(2);
const file = argv.find((arg) => !arg.startsWith("--")); const file = argv.find((arg) => !arg.startsWith("--"));
@@ -128,5 +183,6 @@ if (require.main === module) {
module.exports = { module.exports = {
digest, digest,
digestGltf,
readGlbJson, readGlbJson,
}; };

View File

@@ -2,6 +2,7 @@
const fs = require("fs"); const fs = require("fs");
const path = require("path"); const path = require("path");
const { BUDGETS } = require("./stage-manifest");
function readAreaConfig(file, options = {}) { function readAreaConfig(file, options = {}) {
if (!fs.existsSync(file)) { if (!fs.existsSync(file)) {
@@ -23,6 +24,7 @@ function normalizeAreaConfig(raw, options = {}) {
const areaDir = path.resolve(outputOverrides.areaDir || path.join(outputRoot, id)); const areaDir = path.resolve(outputOverrides.areaDir || path.join(outputRoot, id));
const fileStem = outputOverrides.fileStem || id; const fileStem = outputOverrides.fileStem || id;
const compress = normalizeCompressConfig(raw.compress); const compress = normalizeCompressConfig(raw.compress);
const budget = normalizeBudgetConfig(raw.budget);
const compressedFileStem = outputOverrides.compressedFileStem || const compressedFileStem = outputOverrides.compressedFileStem ||
`${fileStem}-compressed-webp${compress.textureSize}${compress.meshopt ? "-meshopt" : ""}`; `${fileStem}-compressed-webp${compress.textureSize}${compress.meshopt ? "-meshopt" : ""}`;
const pipelineDir = path.resolve(outputOverrides.pipelineDir || path.join(areaDir, "_pipeline")); const pipelineDir = path.resolve(outputOverrides.pipelineDir || path.join(areaDir, "_pipeline"));
@@ -92,10 +94,34 @@ function normalizeAreaConfig(raw, options = {}) {
officeOverrides: raw.blender?.officeOverrides || raw.blender?.office_overrides || "", officeOverrides: raw.blender?.officeOverrides || raw.blender?.office_overrides || "",
}, },
compress, compress,
budget,
outputs, outputs,
}; };
} }
function normalizeBudgetConfig(raw) {
if (raw !== undefined && raw !== null && (typeof raw !== "object" || Array.isArray(raw))) {
throw new Error("budget must be an object");
}
const value = raw || {};
const budget = {
glbBytes: megabytesOption(value.glbSizeMb, BUDGETS.glbBytes, "budget.glbSizeMb"),
glbNodes: integerOption(value.nodes, BUDGETS.glbNodes, "budget.nodes"),
glbImages: integerOption(value.images, BUDGETS.glbImages, "budget.images"),
glbTriangles: integerOption(value.triangles, BUDGETS.glbTriangles, "budget.triangles"),
glbImageBytes: megabytesOption(value.embeddedImageBytesMb, BUDGETS.glbImageBytes, "budget.embeddedImageBytesMb"),
reason: value.reason ?? "",
};
if (typeof budget.reason !== "string") {
throw new Error("budget.reason must be a string");
}
const loosened = Object.keys(BUDGETS).some((key) => budget[key] > BUDGETS[key]);
if (loosened && budget.reason.trim() === "") {
throw new Error("budget.reason is required when a budget exceeds the default");
}
return budget;
}
function requireText(value, key) { function requireText(value, key) {
if (typeof value !== "string" || value.trim() === "") { if (typeof value !== "string" || value.trim() === "") {
throw new Error(`Missing config key: ${key}`); throw new Error(`Missing config key: ${key}`);
@@ -121,6 +147,22 @@ function numberOption(value, fallback, label, min, max) {
return number; return number;
} }
function integerOption(value, fallback, label) {
const number = value === undefined ? fallback : Number(value);
if (!Number.isInteger(number) || number < 1) {
throw new Error(`${label} must be a positive integer`);
}
return number;
}
function megabytesOption(value, fallbackBytes, label) {
const megabytes = value === undefined ? fallbackBytes / 1024 / 1024 : Number(value);
if (!Number.isFinite(megabytes) || megabytes <= 0) {
throw new Error(`${label} must be a positive finite number`);
}
return Math.round(megabytes * 1024 * 1024);
}
function booleanOption(value, fallback, label) { function booleanOption(value, fallback, label) {
if (value === undefined) return fallback; if (value === undefined) return fallback;
if (typeof value === "boolean") return value; if (typeof value === "boolean") return value;

View File

@@ -10,7 +10,14 @@ const {
layerFile, layerFile,
} = require("./scene-layers"); } = require("./scene-layers");
const { digest: glbDigest } = require("../glb-digest"); const { digest: glbDigest } = require("../glb-digest");
const { BUDGETS, fileRecord, readStageManifest, stageManifestPath } = require("./stage-manifest"); const {
BUDGETS,
evaluateGlbBudget,
fileRecord,
glbBudgetWarnings,
readStageManifest,
stageManifestPath,
} = require("./stage-manifest");
function defaultConfigPath(repoRoot) { function defaultConfigPath(repoRoot) {
return path.join(repoRoot, "config", "areas", "nantaizi-lake-innovation-valley.json"); return path.join(repoRoot, "config", "areas", "nantaizi-lake-innovation-valley.json");
@@ -604,15 +611,7 @@ function collectWarnings(area, osm, artifacts, manifests, glb, metadata) {
} }
} }
if (glb) { if (glb) {
if (glb.fileBytes > BUDGETS.glbBytes) { warnings.push(...glbBudgetWarnings("GLB", glb, area.budget).map((warning) => `${warning}.`));
warnings.push(`GLB size ${mb(glb.fileBytes)} MB exceeds budget ${mb(BUDGETS.glbBytes)} MB.`);
}
if (glb.counts.nodes > BUDGETS.glbNodes) {
warnings.push(`GLB nodes ${glb.counts.nodes} exceed budget ${BUDGETS.glbNodes}.`);
}
if (glb.counts.images > BUDGETS.glbImages) {
warnings.push(`GLB images ${glb.counts.images} exceed budget ${BUDGETS.glbImages}.`);
}
} }
if (metadata && metadata.assets < 1) { if (metadata && metadata.assets < 1) {
warnings.push("Cesium metadata has no assets entries."); warnings.push("Cesium metadata has no assets entries.");
@@ -707,6 +706,9 @@ function printDiagnosticsReport(result) {
`${glb.counts.materials} materials, ${glb.counts.images} images, ${glb.counts.accessors} accessors`, `${glb.counts.materials} materials, ${glb.counts.images} images, ${glb.counts.accessors} accessors`,
); );
console.log(` Extensions: ${glb.extensionsUsed.length ? glb.extensionsUsed.join(", ") : "none"}`); console.log(` Extensions: ${glb.extensionsUsed.length ? glb.extensionsUsed.join(", ") : "none"}`);
printGlbBudget(glb, area.budget);
printGlbSources(glb);
printGlbImages(glb);
} }
console.log(""); console.log("");
@@ -789,15 +791,7 @@ function classifyAreaQuality(result) {
} }
if (glb) { if (glb) {
if (glb.fileBytes > BUDGETS.glbBytes) { failures.push(...glbBudgetWarnings("GLB", glb, result.area.budget).map((warning) => `${warning}.`));
failures.push(`GLB size ${mb(glb.fileBytes)} MB exceeds budget ${mb(BUDGETS.glbBytes)} MB.`);
}
if (glb.counts.nodes > BUDGETS.glbNodes) {
failures.push(`GLB nodes ${glb.counts.nodes} exceed budget ${BUDGETS.glbNodes}.`);
}
if (glb.counts.images > BUDGETS.glbImages) {
failures.push(`GLB images ${glb.counts.images} exceed budget ${BUDGETS.glbImages}.`);
}
} }
if (!result.area.blender.treeStyle) { if (!result.area.blender.treeStyle) {
@@ -832,6 +826,35 @@ function formatBytes(bytes) {
return `${bytes} B`; return `${bytes} B`;
} }
function printGlbBudget(glb, budget) {
const result = evaluateGlbBudget(glb, budget);
console.log(" Budget:");
console.log(` Size: ${formatBytes(result.usage.glbBytes)} / ${formatBytes(result.limits.glbBytes)}`);
console.log(` Nodes: ${result.usage.glbNodes} / ${result.limits.glbNodes}`);
console.log(` Images: ${result.usage.glbImages} / ${result.limits.glbImages}`);
console.log(` Render triangles: ${result.usage.glbTriangles} / ${result.limits.glbTriangles}`);
console.log(` Embedded images: ${formatBytes(result.usage.glbImageBytes)} / ${formatBytes(result.limits.glbImageBytes)}`);
if (budget.reason) console.log(` Exception: ${budget.reason}`);
}
function printGlbSources(glb) {
const sources = (glb.sourceSummary || []).slice(0, 5);
if (!sources.length) return;
console.log(" Top sources:");
for (const source of sources) {
console.log(` ${source.source}: ${source.nodes} nodes, ${source.triangles} triangles`);
}
}
function printGlbImages(glb) {
const images = (glb.images || []).filter((image) => image.bytes > 0).slice(0, 5);
if (!images.length) return;
console.log(" Top embedded images:");
for (const image of images) {
console.log(` ${image.name || "unnamed"}: ${formatBytes(image.bytes)}`);
}
}
module.exports = { module.exports = {
BUDGETS, BUDGETS,
analyzeOsmPreflight, analyzeOsmPreflight,

View File

@@ -9,6 +9,8 @@ const BUDGETS = {
glbBytes: 25 * 1024 * 1024, glbBytes: 25 * 1024 * 1024,
glbNodes: 1000, glbNodes: 1000,
glbImages: 24, glbImages: 24,
glbTriangles: 250000,
glbImageBytes: 20 * 1024 * 1024,
}; };
function stageManifestPath(area, stage) { function stageManifestPath(area, stage) {
@@ -64,18 +66,36 @@ function glbSummary(digest) {
} }
function glbBudgetWarnings(label, digest, budgets = BUDGETS) { function glbBudgetWarnings(label, digest, budgets = BUDGETS) {
if (!digest) return []; return evaluateGlbBudget(digest, budgets).violations.map(
const warnings = []; (violation) => `${label} ${violation.label} ${formatBudgetValue(violation.actual, violation.unit)} ` +
if (digest.fileBytes > budgets.glbBytes) { `exceeds budget ${formatBudgetValue(violation.limit, violation.unit)}`,
warnings.push(`${label} GLB size ${mb(digest.fileBytes)} MB exceeds budget ${mb(budgets.glbBytes)} MB`); );
} }
if (digest.counts.nodes > budgets.glbNodes) {
warnings.push(`${label} GLB nodes ${digest.counts.nodes} exceed budget ${budgets.glbNodes}`); function evaluateGlbBudget(digest, budgets = BUDGETS) {
} const usage = {
if (digest.counts.images > budgets.glbImages) { glbBytes: digest?.fileBytes ?? 0,
warnings.push(`${label} GLB images ${digest.counts.images} exceed budget ${budgets.glbImages}`); glbNodes: digest?.counts?.nodes ?? 0,
} glbImages: digest?.counts?.images ?? 0,
return warnings; glbTriangles: digest?.counts?.renderTriangles ?? 0,
glbImageBytes: digest?.embeddedImageBytes ?? 0,
};
const definitions = [
["glbBytes", "GLB size", "bytes"],
["glbNodes", "GLB nodes", "count"],
["glbImages", "GLB images", "count"],
["glbTriangles", "GLB rendered triangles", "count"],
["glbImageBytes", "GLB embedded image bytes", "bytes"],
];
const limits = { ...BUDGETS, ...budgets };
const violations = definitions
.filter(([key]) => usage[key] > limits[key])
.map(([key, label, unit]) => ({ key, label, unit, actual: usage[key], limit: limits[key] }));
return { usage, limits, violations };
}
function formatBudgetValue(value, unit) {
return unit === "bytes" ? `${mb(value)} MB` : String(value);
} }
function sha256(file) { function sha256(file) {
@@ -90,6 +110,7 @@ function mb(bytes) {
module.exports = { module.exports = {
BUDGETS, BUDGETS,
evaluateGlbBudget,
fileRecord, fileRecord,
glbBudgetWarnings, glbBudgetWarnings,
glbSummary, glbSummary,

View File

@@ -0,0 +1,58 @@
#!/usr/bin/env node
"use strict";
const assert = require("assert");
const fs = require("fs");
const os = require("os");
const path = require("path");
const { normalizeAreaConfig } = require("./lib/area-config");
const { digestGltf } = require("./glb-digest");
const { evaluateGlbBudget, BUDGETS } = require("./lib/stage-manifest");
const gltf = {
nodes: [
{ name: "Building_1", mesh: 0 },
{ name: "Shapespark_tree-01", mesh: 1 },
],
meshes: [
{ name: "Building", primitives: [{ indices: 0, attributes: { POSITION: 1 } }] },
{ name: "Tree", primitives: [{ indices: 2, attributes: { POSITION: 3 } }] },
],
accessors: [{ count: 12 }, { count: 4 }, { count: 9 }, { count: 3 }],
images: [{ name: "large", bufferView: 0 }, { name: "small", bufferView: 1 }],
bufferViews: [{ byteLength: 800 }, { byteLength: 200 }],
};
const digest = digestGltf(gltf, { file: "fixture.glb", fileBytes: 1500 });
assert.equal(digest.counts.triangles, 7);
assert.equal(digest.counts.renderTriangles, 7);
assert.equal(digest.embeddedImageBytes, 1000);
assert.deepEqual(digest.sourceSummary.map((entry) => [entry.source, entry.triangles]), [["buildings", 4], ["vegetation", 3]]);
assert.deepEqual(digest.images.map((image) => image.bytes), [800, 200]);
const budget = { ...BUDGETS, glbBytes: 1000 };
assert.equal(evaluateGlbBudget(digest, budget).violations[0].key, "glbBytes");
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), "asset-budget-"));
const input = path.join(tempDir, "input.osm");
fs.writeFileSync(input, "<osm/>");
const base = { id: "test-area", input, outputRoot: tempDir };
assert.equal(normalizeAreaConfig({ ...base, budget: { nodes: 800 } }).budget.glbNodes, 800);
assert.throws(
() => normalizeAreaConfig({ ...base, budget: { nodes: 1200 } }),
/budget.reason is required/,
);
assert.throws(
() => normalizeAreaConfig({ ...base, budget: { triangles: 0 } }),
/budget.triangles must be a positive integer/,
);
assert.throws(
() => normalizeAreaConfig({ ...base, budget: "large" }),
/budget must be an object/,
);
assert.equal(
normalizeAreaConfig({ ...base, budget: { nodes: 1200, reason: "Dense campus vegetation" } }).budget.glbNodes,
1200,
);
fs.rmSync(tempDir, { recursive: true, force: true });
console.log("Asset budget tests passed.");