76 changed files with 1535 additions and 179 deletions

No files matched your search

+3 -1
View File
@@ -4,4 +4,6 @@ outputs/
__pycache__/
assets/models/speedtree/
assets/models/lyrog/
inputs/osm
inputs/osm
inputs/console/
.build-console/
+28
View File
@@ -307,6 +307,34 @@ npm run build:area -- --config config/areas/<area>.json --stages blender,cesium,
| 直接用 Cesium `Model.getMaterial().setValue()` 改普通 glTF PBR 材质 | 运行时数字仍保持原色,不能实现相位字色 |
| 每个信号灯各自生成 0-19 全套倒计时 mesh | 节点和几何按信号数量线性膨胀;应按两个 phase group 共享 |
## Cesium 导出准备:依赖图更新边界
`export_cesium.py` 的 `prepare_independent_meshes(candidates)` 只优化可证明为对象局部的
准备工作。Blender 的 `bpy.ops` 调用前后会更新 view layer;对约 1,744 个对象逐个调用
选择、修改器及模式切换会重复评估整个场景。实测原导出 173 秒中绝大部分用于更新,
采用以下边界后约 16 秒;不能全局 monkey-patch 或禁用 `_view_layer_update`。
- `select_only(obj)` 用选择数据 API;空修改器栈在改变上下文前直接返回。
- 原有修改器仅允许已证明局部的 `BEVEL` 进入预处理;其他类型保持原顺序回退。
- `batch_triangulate_meshes` 使用真正的 Triangulate 修改器及当前 Blender 默认值,
一次评估后通过 `new_from_object(..., preserve_all_data_layers=True)` 获取结果。
必须先收集所有结果,再替换原网格;替换/删除修改器会使 evaluated 数据失效。
共享、形状键、自定义法线、库链接、fake-user 保留网格不进入这个复制路径。
- `mesh_has_multiple_users` 按 Blender `ID_REAL_USERS` 判断:`users - int(use_fake_user)`。
对多个真实用户的网格,原修改器应用本来就失败,提前跳过保持原拓扑与共享关系。
- 缺 UV 的已准备网格在临时小场景逐对象调用原 `smart_project`,通过 `window.scene`
保持编辑模式上下文一致;有 UV 的模型资产不重展。退出必须恢复窗口场景和 view layer,
删除临时场景。无法满足条件时使用原操作路径。
验证使用 `blender/tools/check_export_preparation.py -- --baseline <original_export_cesium.py>`
在真实 Blender 内比对 control、共享网格、形状键、自定义法线、fake user、Bevel 和
Boolean 依赖回退。真实区域还必须比全部主/分类/动态 GLB 的结构摘要、解码后的索引、
坐标、UV、法线、切线、材质与贴图哈希。不要只用顶点数相同证明视觉一致。
Blender 4.5 的 Triangulate 默认 quad method 是 shortest diagonal,并非 BEAUTY。
直接换成 BMesh BEAUTY 会改变拆面和后续 UV,不能称为等价提速。任何替代算法必须先
核对源码默认值和属性转换路径。详见任务 `09-17-cesium-export-performance/research/`。
## 第三方资产导入的源文件边界
第三方 Blend 只属于一次性入库工具的输入,不能成为 `build:area` 或 Cesium 预览的运行时依赖。
+2 -2
View File
@@ -47,8 +47,8 @@ config/areas/<id>.json
│ → 重建 scene.geojson + scene_style.json
│ → _pipeline/stages/reimport.manifest.json
│
├─[blender]───────▶ compile-native-roads.js + Blender + blender/generate_scene.py
│ 默认读 .osm + native-road/
├─[blender]───────▶ native-road ZIP importer + Blender + blender/generate_scene.py
│ 默认读 .osm + validated ZIP cache/
│ → <id>.blend + <id>.png
│ → _pipeline/stages/blender.manifest.json
│
@@ -1,16 +1,16 @@
# Native Road Compiler Contract
# Native Road Package Contract
The native-road package is maintained in the private `road-compiler` repository:
`https://git.app.que01.top/que01/road-compiler`
This host consumes the exact git tag recorded in `package.json` and lockfile. The host owns area configuration normalization and writes `RoadCompilerInput`; the compiler owns the `native-road-package/v1` file contract and CLI. Production code must invoke the installed compiler CLI through `scripts/lib/road-compiler-cli.js`, not import compiler source files or read `packages/road-compiler`.
The host consumes a read-only ZIP exported by the private `road-compiler` repository. The host never installs or invokes the compiler. `nativeRoadPackage` is resolved relative to the area config, hashed, validated, and extracted into `_pipeline/native-road-import/<sha256>/` before native consumers start.
The completion marker is `NATIVE_ROAD_COMPILE_DONE <json>`. `build-area.js` validates the marker's count, JSON payload, area id, output path, process status, and signal before starting Blender.
## Scope / Trigger
This contract applies to native-road compilation from `scripts/build-area.js`, `scripts/compile-native-roads.js`, and the road workbench fresh-compile callback.
This contract applies to native-road consumption from `scripts/build-area.js`, diagnostics, parity, Blender, and native preview generation.
## Signatures
@@ -22,12 +22,13 @@
"08-25-rc-p0-contract-baseline",
"08-25-rc-p1-package-boundary",
"08-25-rc-p2-repo-split",
"08-25-rc-p3-render-separation"
"08-25-rc-p3-render-separation",
"08-26-rc-p5-artifact-consumption"
],
"parent": null,
"relatedFiles": [],
"notes": "",
"notes": "P3 已完成。P5 artifact-only parent 已建立,P5A ZIP export 已在 road-compiler v0.3.0 完成;P5B 宿主 ZIP import 待实施。",
"meta": {
"next_task": "08-25-rc-p4-drawtonomy-ext"
}
}
}
@@ -0,0 +1,3 @@
{"file":".trellis/spec/blender/testing.md","reason":"P3 scene digest parity 是消费迁移的回归门槛。"}
{"file":".trellis/spec/pipeline/native-road-package.md","reason":"验证 ZIP 解压后的 v1.1 contract、areaId 和 source 集合。"}
{"file":".trellis/spec/guides/artifact-parity-guide.md","reason":"验证导入迁移不改变 Blender 产物结构。"}
@@ -0,0 +1,23 @@
# P5 Artifact-Only Road Package Design
P5 replaces the current runtime dependency boundary with a file-only boundary:
```text
road-compiler Web editor
-> deterministic root-flat native-road-package/v1.1 ZIP
-> osmWorkflow config nativeRoadPackage
-> private verified extraction cache
-> Blender / Cesium / preview
```
The road-compiler repository owns export and all editing/compiler internals. osmWorkflow owns ZIP safety,
contract validation, Blender materials and downstream scene construction. Neither repository reads the
other's source, configuration, overrides or runtime implementation.
The ZIP is immutable input and the host's `_pipeline/native-road-import/<sha256>` is disposable cache.
The host rejects unsafe archive topology before writing files, then validates v1.1 `manifest.json` and
area identity before starting Blender. Stage manifests record the ZIP hash and resolved package records.
P5A publishes a fixture/exporter first. P5B uses that fixture to remove compiler CLI installation and
invocation from the host. The migration is intentionally a contract-preserving transport change: rendered
geometry and material behavior must remain at the P3 baseline.
@@ -0,0 +1,3 @@
{"file":".trellis/spec/pipeline/native-road-package.md","reason":"宿主与编译器之间的现有文件契约。"}
{"file":".trellis/spec/pipeline/external-tools.md","reason":"ZIP 导入必须在 Blender 副作用前完成安全校验。"}
{"file":".trellis/spec/guides/cross-layer-thinking-guide.md","reason":"导出、Node 导入、Blender 和 preview 的跨层数据流。"}
@@ -0,0 +1,14 @@
# P5 Integration Plan
1. Complete P5A in `road-compiler`: deterministic Web/CLI ZIP export plus supported-area fixtures and a
tagged compiler release.
2. Complete P5B in `osmWorkflow`: safe importer, config migration, CLI/dependency removal and downstream
descriptor migration.
3. Run the shared end-to-end gate on `fengshu-er-road` and `nantaizi-lake-innovation-valley`:
import ZIP -> Blender -> Cesium -> package -> preview -> area checks.
4. Compare Blender scene digests to P3 control; run negative ZIP tests and legacy provider tests.
5. Record contract docs in both repositories, commit each repository independently, then archive both
children and this integration parent.
Integration rollback: restore the host's previous tagged compiler dependency and compile adapter. ZIP export
is additive and its generated files remain valid standalone artifacts.
@@ -0,0 +1,95 @@
# 道路包导入与纯消费管线
父任务:`.trellis/tasks/08-25-road-compiler-extraction/`
本任务是集成 parent:导出与导入分别在 road-compiler 和宿主仓库实现,只有两个子任务的
端到端 ZIP 契约都通过后才完成。
## Goal
让 road-compiler Web 导出的 `native-road-package/v1.1` 成为 Blender 管线的唯一
native-road 输入。宿主只校验并消费完整道路包,不再安装、调用或理解道路编译器。
用户价值:人工在 road-compiler Web 编辑后,导出道路包即可稳定交给 Blender / Cesium
构建;编辑成果不再通过发布新的编译器代码版本传播。
## Confirmed Facts
- P3 已使 Blender 通过 `manifest.json` 消费 `layers/*.geojson`,并拒绝无效 manifest、
source/file 不一致和未知宿主材质槽。
- 宿主当前仍通过 `scripts/build-area.js` 调用 `scripts/lib/road-compiler-cli.js`;后者安装并
运行 `@osm-asset/road-compiler`,将 `RoadCompilerInput` 写入 `_pipeline/`。
- native Blender 构建当前强制使用 `outputs/<area>/native-road/`,并在每次构建前重新编译;
宿主还持有 `nativeRoad.edgeLines`、`junctionTemplates`、overrides 和 compiler runtime
路径等编译器内部概念。
- `osm2streets` 是显式 legacy/debug provider,应保持现有行为。
- 有效验证区域只有 `fengshu-er-road` 与 `nantaizi-lake-innovation-valley`;
`hanyang-block` 不参与。
## Requirements
### R1 纯产物边界
- native provider 只接收一个完整 `native-road-package/v1.1` 目录。
- 宿主不得依赖 `@osm-asset/road-compiler`,不得调用 compiler CLI,不得写
`RoadCompilerInput`,不得读取或解释 compiler overrides / junction templates。
- 宿主以 manifest 作为道路图层事实源,Blender 不增加 compiler 专用图层表。
### R2 ZIP 导入配置和校验
- 每个 native 区域配置一个道路包 ZIP 路径;路径相对于区域配置文件解析并可指向工作区外的
road-compiler Web 导出文件。
- ZIP 根目录必须直接包含 `manifest.json` 和契约定义的文件;不得接受多余顶层目录、绝对路径、
`..` 路径或符号链接。
- 在启动 Blender 前,宿主将 ZIP 安全解压到其私有 pipeline staging 目录,校验 manifest
契约版本、`areaId` 和当前区域一致,并记录 ZIP hash 和展开文件记录到 stage manifest。
- ZIP 永远只读:宿主不复制、覆盖或修改 road-compiler Web 的导出文件;成功或失败后仅清理
自己的临时解压目录。
- Blender 继续校验 manifest 的图层、source、semantic 角色、材质槽和 splitBy 规则。
### R3 后续阶段
- Cesium、package、preview 和区域诊断从导入道路包读取原生车道中心线、signals 和
其他既有 native-road 输入;不重新编译或复制道路包。
- build stage manifest 将道路包作为外部输入记录,而非宿主输出。
### R4 清理和迁移
- 删除宿主对 compiler npm 包、CLI 封装和编译脚本的生产依赖。
- 删除或替换只测试宿主编译行为的测试;保留 package-contract、Blender 和下游消费测试。
- 更新示例配置、README、pipeline spec 和道路包契约,明确 Web export -> host consume
的单向关系。
## Acceptance Criteria
- [ ] AC1 `package.json` / lockfile 和生产脚本不再引用 `@osm-asset/road-compiler` 或其 CLI。
- [ ] AC2 两个有效区域均可从配置指定的已导出 ZIP 道路包运行 Blender、Cesium、package 和 preview,
构建期间没有 compiler 子进程。
- [ ] AC3 宿主在 Blender 前拒绝缺失包、错误 contract、错误 areaId 和无效 manifest;错误说明
指向导入包和契约,而非 compiler 内部。
- [ ] AC4 Blender scene digest 与 P3 control 基线一致;native semantic 层仍不生成几何。
- [ ] AC5 stage manifest / diagnose / check area 将导入道路包记录为输入并能识别缺失或变更。
- [ ] AC6 `osm2streets` legacy provider 行为未变。
- [ ] AC7 文档和配置将人工编辑传播路径表述为 road-compiler Web export -> host consume。
## Out of Scope
- road-compiler Web 的编辑器功能、认证、部署或 UI 改造。
- 修改道路几何、材质、Blender 建筑/植被/水体渲染。
- 新的 manifest 破坏性版本或 lane graph IR。
- 将道路包上传到远程制品库或实现同步服务。
## Key Decision
- Web 导出的是标准 ZIP;宿主配置直接指向 ZIP 而不是目录。ZIP 是跨仓库的不可变交付物,
宿主只在私有 staging 目录中解压和消费,因而没有导出目录 ownership 或覆盖歧义。
## Subtask Map
| 子任务 | 仓库 | 交付物 | 依赖 |
|---|---|---|---|
| `rc-p5a-zip-export` | road-compiler | Web / CLI 可导出的 root-flat `native-road-package/v1.1` ZIP | 无 |
| `rc-p5b-zip-import` | osmWorkflow | 配置、受控解压、校验和纯消费管线 | p5a 的 ZIP fixture |
跨任务验收:同一个 ZIP fixture 必须被 p5b 的两个有效区域构建消费;宿主不得保留
compiler npm 或 CLI 耦合。
@@ -0,0 +1,29 @@
{
"id": "rc-p5-artifact-consumption",
"name": "rc-p5-artifact-consumption",
"title": "道路包导入与纯消费管线",
"description": "让宿主通过配置指向 road-compiler Web 导出的 ZIP 道路包,只校验和消费该版本化文件契约,不再安装或调用编译器。",
"status": "planning",
"dev_type": null,
"scope": null,
"package": null,
"priority": "P2",
"creator": "dingkang",
"assignee": "dingkang",
"createdAt": "2026-08-26",
"completedAt": null,
"branch": null,
"base_branch": "experiment/road-compiler-rethink",
"worktree_path": null,
"commit": null,
"pr_url": null,
"subtasks": [],
"children": [
"08-26-rc-p5b-zip-import",
"08-26-rc-p5a-zip-export"
],
"parent": "08-25-road-compiler-extraction",
"relatedFiles": [],
"notes": "",
"meta": {}
}
@@ -0,0 +1 @@
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}
@@ -0,0 +1,22 @@
# P5A ZIP Export Design
The compiler owns a pure `exportNativeRoadPackage(result, destination)` function. It writes a
temporary root-flat directory from the already compiled Web/CLI state, validates it with the same
manifest/source validator used by compiler output, then writes a deterministic ZIP. The Web download
handler and CLI are thin callers of this function.
ZIP layout:
```text
manifest.json
compiled.json
diagnostics.json
comparison.json
traffic-signal-assemblies.json
traffic-signals.json
layers/<12 declared sources>.geojson
```
`manifest.generator` is informational only. The host accepts solely the `contract`, `areaId`, and
declared files. ZIP creation normalizes entry order and timestamps so the export is hash-stable.
Failure never replaces a chosen output file.
@@ -0,0 +1 @@
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}
@@ -0,0 +1,10 @@
# P5A Execution Plan
1. Inspect workbench state ownership and compiler output writer; keep export sourced from the current
compiled state, not host paths.
2. Add deterministic root-flat ZIP writer plus validation and CLI entry point.
3. Add Web download action using the same writer.
4. Add fixtures for root layout, content determinism, manifest/source parity and edited-state export.
5. Export both supported-area fixtures for P5B, run compiler tests, then publish a compiler release.
Rollback: remove the export entry point; no host behavior changes in this subtask.
@@ -0,0 +1,34 @@
# Web 导出原生道路 ZIP 包
## Goal
让 road-compiler Web/CLI 以 root-flat native-road-package/v1.1 ZIP 导出人工编辑后的道路包。
## Requirements
- 从 Web 当前编辑状态导出完整道路包,而不是源代码、npm 包或仅 overrides。
- ZIP 根直接包含 `manifest.json`,并包含全部 manifest 声明的 `layers/<source>.geojson`、
`compiled.json`、`traffic-signals.json` 和 preview 所需的 signal / diagnostics 文档。
- `manifest.json` 的 `contract` 保持 `native-road-package/v1.1`;新增可选 `generator` 元数据
记录应用版本和 revision,但不得让宿主依赖该版本。
- Web 提供用户可见的下载操作;CLI 提供等价、可测试的 ZIP 导出入口。
- ZIP 条目顺序、JSON 序列化和压缩时间戳必须确定性,重复导出未编辑内容的文件 hash 一致。
## Acceptance Criteria
- [x] 从 Web 编辑后的区域可下载一个 root-flat ZIP,且无需宿主或 npm 发布。
- [x] CLI fixture 导出 ZIP 后,解压内容通过 compiler 的 `validatePublishedLayers()`。
- [x] ZIP 具有恰好一份 manifest 声明的 layer source,无额外 compiler source / overrides 文件。
- [ ] 两个有效区域导出的 ZIP 可被 p5b 导入并完成 Blender/Cesium/preview 构建。
- [x] 不修改 `compiled.json` 与 v1.1 manifest 的既有渲染语义。
## Out of Scope
- 远程存储、上传、项目历史或多用户协作。
- 修改道路编辑器交互、几何算法或 manifest 主版本。
## Key Decisions
- ZIP 是单个标准交付物,根目录平铺 package 文件,不包裹 area-id 顶层目录。
- ZIP 面向宿主消费,只携带已编译渲染/预览数据;编辑源、OSM 与 overrides 留在 road-compiler
项目中,不泄露为宿主依赖。
@@ -0,0 +1,26 @@
{
"id": "rc-p5a-zip-export",
"name": "rc-p5a-zip-export",
"title": "Web 导出原生道路 ZIP 包",
"description": "让 road-compiler Web/CLI 以 root-flat native-road-package/v1.1 ZIP 导出人工编辑后的道路包。",
"status": "in_progress",
"dev_type": null,
"scope": null,
"package": null,
"priority": "P2",
"creator": "dingkang",
"assignee": "dingkang",
"createdAt": "2026-08-26",
"completedAt": null,
"branch": null,
"base_branch": "experiment/road-compiler-rethink",
"worktree_path": null,
"commit": null,
"pr_url": null,
"subtasks": [],
"children": [],
"parent": "08-26-rc-p5-artifact-consumption",
"relatedFiles": [],
"notes": "",
"meta": {}
}
@@ -0,0 +1 @@
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}
@@ -0,0 +1,19 @@
# P5B ZIP Import Design
`scripts/lib/native-road-package.js` becomes the single host adapter. It resolves the config-relative
ZIP path, hashes it, validates central-directory entry names and metadata, and extracts only accepted
regular files to `_pipeline/native-road-import/<sha256>/`. It validates the unpacked manifest and
returns an immutable package descriptor:
```text
{ zipFile, zipRecord, rootDir, manifest, records }
```
The descriptor replaces `area.outputs.nativeRoadDir` as native input for Blender, native preview traffic,
diagnostics and stage manifests. The configured ZIP and unpacked cache are inputs, never published area
outputs. Cache invalidation is by ZIP SHA-256; cleanup only deletes importer-owned hash directories.
The importer rejects any non-root-flat layout, duplicate path, symlink, unsupported compression/error,
path traversal, invalid manifest, area mismatch or declared/published mismatch before Blender starts.
The native compiler CLI dependency and host-owned `nativeRoad` compilation options are removed. Legacy
`osm2streets` remains an explicit separate provider.
@@ -0,0 +1 @@
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}
@@ -0,0 +1,12 @@
# P5B Execution Plan
1. Add config normalization for `nativeRoadPackage`, migrate two supported configs and the template.
2. Add the tested ZIP importer with bounded extraction, root-flat/path/symlink validation, SHA cache and
manifest/area validation.
3. Route Blender, preview traffic, stage manifests and diagnostics through the imported descriptor.
4. Delete compiler CLI/compile production paths and package dependency; replace their tests with ZIP fixture
and negative importer tests.
5. Update docs/specs, run native/legacy unit suites, build two exported packages, then run Blender parity.
Rollback: restore the old compiler dependency and native compile path in one revert; ZIP exports remain
non-destructive standalone files.
@@ -0,0 +1,40 @@
# ZIP 道路包安全导入与纯消费
## Goal
让 osmWorkflow 配置、校验、解压并消费 road-compiler Web 导出的 ZIP 道路包,移除 compiler runtime 耦合。
## Requirements
- 区域配置以 `nativeRoadPackage` 指向 ZIP,路径相对于该区域配置文件;native provider 不再
接受 compiler options、overrides 或输出目录作为输入。
- 宿主在任何 Blender / Cesium 副作用前,验证 ZIP 文件、条目数量/大小、root-flat 文件路径、
无重复路径、无符号链接、无绝对路径与 `..` 路径。
- 仅将已验证的文件解压至 `<area>/_pipeline/native-road-import/<zip-sha256>/`;解压目录是宿主
私有缓存,绝不写回 ZIP 或 road-compiler 导出目录。
- 共享 importer 为 Blender、preview traffic、diagnose/check 和 stage manifest 提供同一包路径与
文件记录;不能让下游自行按字符串拼接另一份 native-road 目录。
- 移除 compiler package、CLI wrapper、compile script、编译器输入配置、以及只覆盖旧调用链的测试。
## Acceptance Criteria
- [ ] 两个有效区域均从 ZIP 完成 `blender,cesium,package,preview`,构建日志没有 compiler CLI。
- [ ] 缺 ZIP、错误 ZIP、nested root、path traversal、symbolic link、错误 contract、错误 areaId 和
manifest/source 不匹配均在启动 Blender 前失败。
- [ ] stage manifest、`diagnose:area` 与 `check:area` 记录 ZIP hash 和已验证的导入目录;ZIP
修改后会被判 stale。
- [ ] `package.json`、lockfile、生产脚本与宿主测试不再引用 `@osm-asset/road-compiler`。
- [ ] Blender digest 对 P3 control 基线一致;legacy `osm2streets` 路径的单元测试不变。
## Out of Scope
- p5a 的 ZIP 生成与 Web 下载实现。
- ZIP 远程下载、解密、签名验证或制品库管理。
- 改道路内容、材质或 legacy QGIS 行为。
## Key Decisions
- ZIP 是只读输入;宿主缓存的生命周期和目录 ownership 完全属于 `_pipeline/`。
- ZIP 解码由明确声明的 Node ZIP 库完成,而不是依赖系统 `unzip`;这使路径/符号链接校验可控并
可在单元测试中覆盖。
- 只有 `native-road-package/v1.1` 可被导入;契约升级必须另开兼容任务。
@@ -0,0 +1,26 @@
{
"id": "rc-p5b-zip-import",
"name": "rc-p5b-zip-import",
"title": "ZIP 道路包安全导入与纯消费",
"description": "让 osmWorkflow 配置、校验、解压并消费 road-compiler Web 导出的 ZIP 道路包,移除 compiler runtime 耦合。",
"status": "in_progress",
"dev_type": null,
"scope": null,
"package": null,
"priority": "P2",
"creator": "dingkang",
"assignee": "dingkang",
"createdAt": "2026-08-26",
"completedAt": null,
"branch": null,
"base_branch": "experiment/road-compiler-rethink",
"worktree_path": null,
"commit": null,
"pr_url": null,
"subtasks": [],
"children": [],
"parent": "08-26-rc-p5-artifact-consumption",
"relatedFiles": [],
"notes": "",
"meta": {}
}
@@ -3,7 +3,7 @@
"name": "rc-p3-render-separation",
"title": "Phase 3:道路与建筑渲染分离",
"description": "编译器输出 layer manifest 自声明图层,Blender 改为读 manifest;道路渲染从 generate_scene.py 抽离,消除跨仓库重复图层表",
"status": "in_progress",
"status": "completed",
"dev_type": null,
"scope": null,
"package": null,
@@ -11,7 +11,7 @@
"creator": "dingkang",
"assignee": "dingkang",
"createdAt": "2026-08-25",
"completedAt": null,
"completedAt": "2026-08-26",
"branch": null,
"base_branch": "main",
"worktree_path": null,
@@ -0,0 +1,4 @@
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}
{"file":".trellis/spec/blender/asset-generation.md","reason":"Asset compatibility"}
{"file":".trellis/spec/guides/artifact-parity-guide.md","reason":"Parity acceptance"}
{"file":".trellis/tasks/09-17-cesium-export-performance/research/triangulation-semantics.md","reason":"Source-backed optimization correctness"}
@@ -0,0 +1,15 @@
# Design
## Evidence and theory
Baseline profiler: 179.6 s total, 166 s in 6,917 automatic view-layer updates; all eight exports 4.4 s. Blender `bpy/ops.py` updates the view layer before and after operators. Object-local mesh preparation should not repeatedly evaluate unrelated scene objects. Modifier evaluation may depend on scene relations, so disabling update globally is unacceptable.
A previous direct-BMesh experiment used BEAUTY for quads, unlike Blender's Triangulate modifier SHORTEDGE default. Both use BM_mesh_triangulate. Match parameters and mesh attribute conversion before accepting a direct path; retain original behavior for shared meshes and unsupported custom data. Research official modifier conversion code to explain residual differences.
## Implementation boundaries
Only exporter mesh preparation plus focused verification tools. Early-return for no-op work before selecting objects. Prefer public data APIs for object-local operations; preserve scene evaluation for actual dependent modifiers. Keep original triangulation modifier semantics via batched evaluation if direct BMesh cannot prove equivalence. No monkeypatching Blender internals in production.
## Compatibility and rollback
Same CLI, completion markers, GLB set, ordering, metadata, materials, and source blend. Validation uses temporary output directories. Reject alternatives that alter geometry/UV/normals without an explained contract. Each optimization separately measurable and removable. Existing user changes outside the exporter remain untouched.
@@ -0,0 +1,4 @@
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}
{"file":".trellis/spec/blender/asset-generation.md","reason":"Deterministic assets and shared mesh contract"}
{"file":".trellis/spec/guides/artifact-parity-guide.md","reason":"Output parity and control requirements"}
{"file":".trellis/tasks/09-17-cesium-export-performance/research/triangulation-semantics.md","reason":"Official source proof and safe alternatives"}
@@ -0,0 +1,11 @@
# Execution
- [x] Preserve original exporter and repeat unmodified control on Nantaizi without profiler; baseline second real area.
- [x] Record Blender source/API proof for proposed replacement operations, including triangulation defaults and conversion flags.
- [x] Implement exporter optimization through Trellis implement agent; main agent owns measurement and independent artifact comparisons.
- [x] Validate all GLBs against baseline and control; compare decoded geometry/attributes/images when digest is insufficient.
- [x] Run real Blender edge fixtures and existing pure Python tests.
- [x] Run fresh final benchmarks and cross-agent check; fix any findings.
- [x] Record results/spec contracts and report measured gain plus limitations. No automatic commit.
User approval: previous turn proposed preserving assets while eliminating redundant updates/modifier work; current user explicitly approved, with theory required. This records that accepted scope rather than requesting approval again.
@@ -0,0 +1,25 @@
# Cesium 导出性能优化与产物一致性验证
## Goal
将 Cesium 导出阶段从约 150 秒降至接近 15–20 秒,保持全部资产与视觉语义。用户在性能诊断之后明确同意优化,并要求每项操作有理论依据。
## Requirements
- 每项优化对应测量热点、Blender API/源码依据及可证伪的验证。
- 保留主场景、分类资产、三个动态信号资产、材质、贴图、节点与地理锚点。
- 不全局关闭依赖图更新;保留涉及修改器依赖的正确评估。
- 不降低几何精度、不改材质、删资产或修改浏览器流程。
- 不覆盖用户现有工作区修改;性能及 parity 产物写临时目录。
## Acceptance Criteria
- [x] 同一输入、无 profiler 的前后基准,记录实际耗时与提升倍率;接近 10 倍是目标,不能牺牲正确性。
- [x] 全部 GLB 通过既有结构摘要契约;深入检查几何、UV、法线、切线及纹理,不能通过扩大忽略名单隐藏差异。
- [x] 至少两份真实区域输入及针对共享网格、无修改器、既有 UV 的 Blender 验证。
- [x] 未改动代码 control 复测;相关现有测试通过。
- [x] 理论依据、验证结果、限制记录到任务研究与长期规范。
## Results
Nantaizi 173.227 → 15.905 秒(10.89x);Fengshu 36.891–41.396 → 5.638 秒(6.54–7.34x)。全部结构摘要通过。Nantaizi 解码属性零差异;Fengshu 仅树冠切线存在未改代码 control 已复现的一个四位小数舍入步长波动。详见 `research/measurements.md`,未扩展项目忽略名单。
@@ -0,0 +1,74 @@
// Diagnostic: run from repository root with two export directory arguments.
// Strict 1e-6 comparison; known control noise is reported, never suppressed.
const fs = require('fs');
const path = require('path');
const assert = require('assert/strict');
const crypto = require('crypto');
const {digest} = require(path.join(process.cwd(), 'scripts/glb-digest'));
function glb(file) {
const data = fs.readFileSync(file);
let json, bin;
for (let i=12; i<data.length;) {
const len=data.readUInt32LE(i), type=data.readUInt32LE(i+4);
if(type===0x4e4f534a) json=JSON.parse(data.subarray(i+8,i+8+len));
if(type===0x004e4942) bin=data.subarray(i+8,i+8+len);
i+=8+len;
}
function accessor(id) {
const a=json.accessors[id];
assert(!a.sparse, 'sparse accessor unsupported');
const v=json.bufferViews[a.bufferView];
const n={SCALAR:1,VEC2:2,VEC3:3,VEC4:4,MAT4:16}[a.type];
const [bytes,fn]={5120:[1,'readInt8'],5121:[1,'readUInt8'],5122:[2,'readInt16LE'],5123:[2,'readUInt16LE'],5125:[4,'readUInt32LE'],5126:[4,'readFloatLE']}[a.componentType];
const out=[];
for(let i=0;i<a.count;i++) for(let j=0;j<n;j++) {
out.push(bin[fn]((v.byteOffset||0)+(a.byteOffset||0)+i*(v.byteStride||n*bytes)+j*bytes));
}
return out;
}
function imageHash(image) {
const v=json.bufferViews[image.bufferView];
return crypto.createHash('sha256').update(bin.subarray(v.byteOffset||0,(v.byteOffset||0)+v.byteLength)).digest('hex');
}
return {json, accessor, imageHash};
}
const [before,after]=process.argv.slice(2);
let failures=0;
for(const file of fs.readdirSync(before).filter(f=>f.endsWith('.glb')).sort()) {
const issues=[];
try {
const da=digest(path.join(before,file)), db=digest(path.join(after,file));
for(const d of [da,db]) {delete d.fileBytes;delete d.buffers;delete d.counts.accessors;}
try {assert.deepEqual(da,db);} catch {issues.push('structural digest differs');}
const a=glb(path.join(before,file)),b=glb(path.join(after,file));
for(const key of ['nodes','scenes','scene','materials','textures','samplers','animations','skins','extensions','extensionsUsed','extensionsRequired']) {
assert.deepEqual(a.json[key],b.json[key],key);
}
assert.equal(a.json.meshes.length,b.json.meshes.length);
let maxError=0;
for(let m=0;m<a.json.meshes.length;m++) {
const am=a.json.meshes[m],bm=b.json.meshes[m];
assert.equal(am.name,bm.name);
assert.equal(am.primitives.length,bm.primitives.length);
for(let p=0;p<am.primitives.length;p++) {
const ap=am.primitives[p],bp=bm.primitives[p];
assert.equal(ap.material,bp.material);
assert.deepEqual(Object.keys(ap.attributes),Object.keys(bp.attributes));
for(const key of ['indices',...Object.keys(ap.attributes)]) {
const av=a.accessor(key==='indices'?ap.indices:ap.attributes[key]);
const bv=b.accessor(key==='indices'?bp.indices:bp.attributes[key]);
if(av.length!==bv.length) {issues.push(`${am.name}/${p}/${key}: count ${av.length} -> ${bv.length}`);continue;}
let error=0;
for(let i=0;i<av.length;i++) error=Math.max(error,Math.abs(av[i]-bv[i]));
maxError=Math.max(maxError,error);
if(error>(key==='indices'?0:1e-6)) issues.push(`${am.name}/${p}/${key}: max delta ${error}`);
}
}
}
assert.equal(a.json.images?.length||0,b.json.images?.length||0);
for(let i=0;i<(a.json.images||[]).length;i++) assert.equal(a.imageHash(a.json.images[i]),b.imageHash(b.json.images[i]));
console.log(file, issues.length?'FAIL':'PASS', 'max attribute delta',maxError,issues.slice(0,12));
} catch(e) {issues.push(e.message);console.log(file,'ERROR',e.message.slice(0,500));}
if(issues.length) failures++;
}
process.exitCode=failures?1:0;
@@ -0,0 +1,48 @@
# Performance and parity evidence (2026-09-17)
Environment: local Blender 4.5.12, same existing `.blend` per region, serial runs, all outputs in `/private/tmp/cesium-profile.9ODFNa`. Original exporter preserved there as `original_export_cesium.py`. No source scene or existing output overwritten.
## Measurements
| Input / implementation | Export function wall time |
| --- | ---: |
| Nantaizi original cProfile | 179.596 s |
| Nantaizi original control without profiler | 173.227 s |
| Nantaizi initial production candidate | 33.389 s |
| Nantaizi candidate with shared-mesh guards | 15.337 s |
| Nantaizi final fake-user guard correction | 15.905 s |
| Fengshu original | 41.396 s |
| Fengshu original second control | 36.891 s |
| Fengshu optimized | 5.527 s |
| Fengshu final fake-user guard correction | 5.638 s |
Time excludes Blender process startup; `measure.py` uses perf_counter around export() and per-function timing wrappers, no cProfile. Inputs/outputs and eight/seven export calls unchanged. Final Nantaizi speedup 10.89x; Fengshu 6.54–7.34x. Final Nantaizi eight GLBs again pass with zero decoded attribute delta. Both regions' metadata matches after normalizing output directory prefixes.
## Nantaizi parity
Unchanged control compared with original profiler output: all eight GLBs have identical project structural digests (existing unstable fields excluded), decoded indices/positions/UVs/normals/tangents/other attributes, full nodes/scenes/materials/textures/samplers and embedded image hashes. Maximum numeric delta 0. Initial and shared-guard production candidates both pass these exact comparisons.
## Fengshu control noise
All seven project structural digests match. Six GLBs' decoded attributes also match exactly. `vegetation.glb`, mesh `Tree_Natural_Crowns_DarkMesh`, TANGENT values vary by at most 0.00010001659393310547 between two **unmodified** runs. Optimized versus first baseline has the same maximum; versus second baseline 0.00010000169277191162. Positions, UVs, normals, indices, other meshes and embedded images match. This is not a broad tolerance waiver.
Installed `io_scene_gltf2/blender/exp/primitive_extract.py:1507` rounds tangents to `ROUNDING_DIGIT`, defined as 4 in `io/com/constants.py:159`. A 0.0001 difference is one exporter quantization step. Existing project digest ignores accessor/buffer layout noise, but no project ignore list was expanded for this work.
## Edge fixtures
`blender/tools/check_export_preparation.py` passed in real Blender using original exporter as reference and an unchanged control. Includes asymmetric quad/ngon triangle counts, authored UVs, shared meshes, custom normals, shape keys, Bevel, dependent Boolean, and fake retention user. Compares topology/vertex order/UV/normals/material slots/mesh names/custom properties/sharing/remaining modifiers. Uses 1e-6 numerical tolerance.
Independent review moved edit-mode selection into the cleanup try/finally and added an injected UV failure fixture asserting restoration of original scene/view layer/collection links and no scene leak. Expanded fixture passed after review. All 64 existing pure Python tests, Python compilation and `git diff --check` pass. No project Python lint/type-check configuration exists. No UI screenshot claim: visual semantics checked through decoded render attributes and identical image bytes; no browser rendering code changed.
Reproducible strict comparison: from repo root run `node .trellis/tasks/09-17-cesium-export-performance/research/compare-glb.cjs <baseline-directory> <candidate-directory>`. It reports the Fengshu control tangent noise as a failure at 1e-6, deliberately not masking it. Classify that one known result using the unchanged control evidence above; all project structural digest checks pass.
## Source-backed choices
- Public selection data API avoids operator entry/exit graph updates.
- Empty modifier stack returns before any selection/update.
- Same Triangulate modifier is evaluated in a batch; evaluated meshes copied before mutating original data; preserve all data layers. No BMesh approximation.
- Same UV operator runs one object at a time in a small temporary scene, switched through actual window.scene; normal graph updates stay enabled.
- Shared triangulation skipped using `ID_REAL_USERS = users - fake_user`, matching Blender's failure precondition.
- Unknown modifier stacks retain original ordering; no global monkey patch.
Failed temp_override-only UV investigation was stopped because Blender edit-mode poll requires consistent window scene context; production uses a real temporary scene/window switch instead. Diagnostic global update suppression and mismatched direct-BMesh experiment are not in production.
@@ -0,0 +1,82 @@
# Research: Triangulation semantics and output-preserving optimization
- Query: Explain the temporary direct-BMesh experiment's different GLB vertex counts and identify source-backed faster alternatives.
- Scope: mixed (project source, installed exporter, official Blender 4.5.0 source)
- Date: 2026-09-17
## Findings
### Confirmed algorithm mismatch in the experiment
`blender/export_cesium.py:531` creates a Triangulate modifier and only overrides `min_vertices = 4`. The temporary `/private/tmp/cesium-profile.9ODFNa/profile_export.py:42` instead passes `quad_method='BEAUTY', ngon_method='BEAUTY'` to BMesh.
Blender **4.5.0** `source/blender/makesdna/DNA_modifier_defaults.h:643-649` defines the modifier defaults as:
```
flag = 0
quad_method = MOD_TRIANGULATE_QUAD_SHORTEDGE
ngon_method = MOD_TRIANGULATE_NGON_BEAUTY
min_vertices = 4
```
Thus the experiment is not algorithm-equivalent: quads use shortest diagonal in the baseline and beauty in the experiment. This is a concrete reason to correct the experiment, not speculative parameter tuning. Do not rely on older Blender documentation's defaults; read a temporary modifier's RNA properties on the actual running version if supporting multiple Blender versions.
Both routes ultimately call `BM_mesh_triangulate` (official `MOD_triangulate.cc`, `bmo_triangulate.cc:29-44`). The modifier passes `tag_only=false`; the BMesh operator tags its supplied faces and passes `tag_only=true`. Supplying all eligible faces and matching quad/ngon methods aligns the core algorithm.
### Why GLB vertex counts can differ with equal triangles
Installed `/Applications/Blender.app/Contents/Resources/4.5/scripts/addons_core/io_scene_gltf2/blender/exp/primitive_extract.py:336-344` explicitly explains that Blender corner attributes are packed into per-vertex records for glTF. Deduplication at lines 825 and 900-901 uses the attribute record, not position alone. Normals, tangents and UV discontinuities therefore split vertices without changing triangle count. Changing the quad diagonal can change subsequent normals and UV unwrap even when all positions and triangle totals remain unchanged.
Project preparation order (`blender/export_cesium.py:670-676`) is triangulate then unwrap. Hence changing topology before `smart_project` can also change UV islands. Counts alone do not establish visual parity or identify which attribute actually changed in the existing artifacts; inspect decoded attribute arrays per mesh to attribute the observed count difference definitively.
### Remaining BMesh conversion differences
Official `MOD_triangulate.cc` sets `calc_face_normal=true`, `calc_vert_normal=false` and carries original-index custom-data masks through conversion. Python `BMesh.from_mesh` (`bmesh_py_types.cc:1376-1414`) defaults both normal flags to true; pass `face_normals=True, vertex_normals=False` to match the modifier's inputs more closely.
Modifier `flag=0` means **keep_custom_normals is false by default in 4.5.0**. Therefore do not claim custom-normal preservation is the established reason for this scene's discrepancy. If enabled, the modifier explicitly copies evaluated corner normals into a temporary CD_NORMAL layer and restores normalized custom normals after triangulation. Direct BMesh does not reproduce that explicit procedure automatically.
Python `BMesh.to_mesh` (`bmesh_py_types.cc:1201-1240`) uses `BM_mesh_bm_to_me`, updates shape-key indices/object remaps and tags geometry dirty; modifier output uses `BKE_mesh_from_bmesh_for_eval_nomain`. These are distinct conversion paths. Matching triangulation settings does not constitute a proof of byte-identical mesh output, ordering or custom data. Preserve the baseline fallback for multi-user meshes (`modifier_apply` refuses them); do not change shared topology as a performance side effect.
### Strongest semantic route: keep the actual modifier, batch evaluation
For simple single-user meshes without pre-existing modifiers or shape keys, add the same Triangulate modifier through the data API for all eligible objects, evaluate the dependency graph once, then obtain persistent meshes with `bpy.data.meshes.new_from_object(obj.evaluated_get(depsgraph), preserve_all_data_layers=True, depsgraph=depsgraph)`. Stage every result before replacing source data/removing modifiers, because mutation can invalidate evaluated state. This executes the actual modifier implementation instead of duplicating it in Python.
Official `rna_main_api.cc:1103-1123` says an original object produces undeformed geometry while an evaluated object produces final geometry with all modifiers; preserving all data layers requires a dependency graph. Official `rna_ID.cc:2447-2452` says `evaluated_get` only retrieves the **last** evaluation and does **not** ensure the graph is up to date. An explicit graph evaluation barrier after adding modifiers is mandatory; repeated evaluation per object defeats the optimization.
Limit this fast path to proven equivalent cases. Existing modifier stacks, hidden modifiers, failing modifiers, shape keys, shared meshes and inter-object dependencies are not generally equivalent to the current sequential `modifier_apply` loop. Keep the existing operator fallback for those cases until separately justified. New datablocks must preserve material slots, required custom properties and semantic names; parity must detect unintended instance splitting or GLB naming differences.
### Recommended decision
1. Correct the demonstrated quad-method mismatch before drawing conclusions from direct-BMesh measurements.
2. Prefer batched evaluation of the original modifier if exact parity remains the requirement; its geometric semantics have a stronger equivalence basis.
3. Keep graph updates at real dependency boundaries; never ship the temporary global `_view_layer_update` monkey patch.
4. Require decoded triangle attributes and existing project structure parity, including UVs/normals/tangents and dynamic/shared assets. Do not accept matching counts as sufficient evidence.
## Files found
- `blender/export_cesium.py` — preparation order and baseline modifier/UV operations.
- `/private/tmp/cesium-profile.9ODFNa/profile_export.py` — temporary monkey-patch and direct-BMesh experiment.
- `/Applications/Blender.app/Contents/Resources/4.5/scripts/addons_core/io_scene_gltf2/blender/exp/primitive_extract.py` — glTF corner-attribute splitting and deduplication.
## External references
All source references pinned to Blender `v4.5.0`:
- https://github.com/blender/blender/blob/v4.5.0/source/blender/makesdna/DNA_modifier_defaults.h#L643
- https://github.com/blender/blender/blob/v4.5.0/source/blender/modifiers/intern/MOD_triangulate.cc
- https://github.com/blender/blender/blob/v4.5.0/source/blender/bmesh/operators/bmo_triangulate.cc
- https://github.com/blender/blender/blob/v4.5.0/source/blender/python/bmesh/bmesh_py_types.cc#L1376
- https://github.com/blender/blender/blob/v4.5.0/source/blender/makesrna/intern/rna_main_api.cc#L1103
- https://github.com/blender/blender/blob/v4.5.0/source/blender/makesrna/intern/rna_ID.cc#L2447
## Related specs
- `.trellis/spec/blender/index.md` — Blender/standard-Python boundary and parity requirements.
- `.trellis/spec/blender/asset-generation.md` — deterministic geometry, shared meshes and dynamic export assets.
- `.trellis/spec/blender/testing.md` — mathematical expectations and real Blender parity.
## Caveats / Not Found
- Research only: no Blender runs or changes to product code were performed.
- The algorithm mismatch is proven from source; the exact per-attribute cause of every previously observed vertex-count change is not measured here.
- Batch evaluated-mesh conversion is a source-backed candidate, not a claim of verified output parity or measured speedup.
@@ -0,0 +1,26 @@
{
"id": "cesium-export-performance",
"name": "cesium-export-performance",
"title": "Cesium 导出性能优化与产物一致性验证",
"description": "",
"status": "completed",
"dev_type": null,
"scope": null,
"package": null,
"priority": "P2",
"creator": "dingkang",
"assignee": "dingkang",
"createdAt": "2026-09-17",
"completedAt": "2026-09-17",
"branch": null,
"base_branch": "experiment/road-compiler-rethink",
"worktree_path": null,
"commit": null,
"pr_url": null,
"subtasks": [],
"children": [],
"parent": null,
"relatedFiles": [],
"notes": "",
"meta": {}
}
@@ -0,0 +1 @@
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}
@@ -0,0 +1,60 @@
# 本地构建控制台设计
## Architecture
新增 `npm run console` 启动的本机控制服务。服务只绑定 `127.0.0.1`,启动后打开默认浏览器;它以 Node 内置 `http` 提供 JSON API、Server-Sent Events(SSE)和无构建的静态 HTML/CSS/JS。首版不引入 React、Vite、Electron 或数据库。
```text
browser UI
│ HTTP / SSE, localhost only
▼
scripts/build-console.js
├─ configs + diagnostics + stage manifests → area/history cards
├─ import OSM/ZIP → inputs/console/<config-key>/ + ownership record
├─ spawn node scripts/build-area.js → streamed build log/job state
├─ serve outputs/<area-dir>/ as read-only preview assets
└─ validated cleanup target → macOS Trash
```
`build-area.js` continues to own all pipeline stages. The console does not import it in-process or duplicate its business logic; it starts it as a child process with `--config` and optional `--stages`, collecting stdout/stderr for the UI.
## Data and ownership
At startup the service enumerates `config/areas/*.json`, calls the existing `readAreaConfig()` / diagnostics APIs, and reads stage manifests. Those files are the history source of truth.
The import form copies selected OSM and ZIP files into `inputs/console/<config-key>/`. A local ignored ownership record under `.build-console/areas/<config-key>.json` records the configuration filename, managed input paths, ZIP hash, and output directory. It exists only to prove which files the console may clean; ordinary historical configs with external paths remain usable but do not grant deletion authority over those sources.
The ZIP `areaId` is immutable and becomes configuration `id`. Because an explicit Save As must not collide with its source config or generated output, the user-facing configuration key is a separate filesystem-safe slug; a saved copy gets a distinct config filename and explicit `outputs.areaDir` / `fileStem` derived from that key. The package contract keeps the original ZIP `areaId`.
## User flow
1. **Welcome / readiness**: display a short three-step guide and checks for Node, Blender, writable managed input directory, and existing areas. Explain that OSM supplies buildings/landscape while ZIP supplies authoritative roads.
2. **History**: show every `config/areas/*.json` as a card with input locations, tree style, output health, last successful stage and links to preview/package when present.
3. **Import / Save As**: choose OSM and ZIP, inspect ZIP contract and locked `areaId`, choose tree style, select a config key, then copy inputs and atomically write a new config. Existing config edits default to Save As; overwrite requires a typed/explicit confirmation.
4. **Build**: choose standard full build or individual existing stages. At most one build job may run; SSE reports stdout/stderr, named stage progression, completion marker and a concise actionable error summary.
5. **Preview**: link to a read-only `/artifacts/<config-key>/...` route that serves the generated preview HTML and its package-relative assets. Vehicle selection stays in the existing preview runtime, rather than changing build artifacts.
6. **Cleanup**: cards and multi-select provide (a) Clear build outputs and (b) Remove area. A confirmation screen lists regions, file counts and byte totals. Targets must be canonically located under allowed project roots. macOS Finder moves them to Trash; it never calls recursive shell deletion.
## API boundary
- `GET /api/health`, `/api/areas`, `/api/options`: readiness, history cards and supported tree styles.
- `POST /api/import/inspect`: multipart/selected file validation without promoting files; verifies ZIP root/contract/areaId via the shared native package validator.
- `POST /api/areas`: copy managed inputs, write ownership record and new config atomically.
- `POST /api/areas/:key/build`: create one build job; `GET /api/jobs/:id/events` streams SSE logs and terminal state.
- `POST /api/cleanup/preview` and `/api/cleanup/confirm`: separate target computation from confirmation, avoiding a stale/hidden destructive scope.
- `GET /artifacts/:key/*`: safely serve only the selected area output root; reject traversal and arbitrary filesystem paths.
Requests are same-origin, localhost-only and JSON validated. File names are normalized to generated safe names; filesystem writes use staging plus rename. Cleanup uses `realpath` containment checks immediately before moving targets to Trash.
## Compatibility and rollout
- Existing commands and hand-authored configurations remain unchanged.
- Existing configs are visible even when their input paths are external; they are never copied or marked owned until a user explicitly imports/saves a console-managed version.
- The console's new local state and managed imports are gitignored. Normal area config files remain reviewable JSON.
- Rollback is deleting the new console-specific files and npm script; it never changes pipeline output contracts.
## Validation strategy
- Unit tests for config/history discovery, ZIP inspection, ownership classification, safe path rejection, Save As collision handling, cleanup plan creation and destructive-target containment.
- HTTP tests for localhost binding, API validation, SSE job lifecycle and artifact traversal rejection.
- Manual smoke test: import OSM+ZIP, save, full build, preview, clear outputs, rebuild, remove area, restore from Trash if needed.
@@ -0,0 +1 @@
{"_example": "Fill with {\"file\": \"<path>\", \"reason\": \"<why>\"}. Put spec/research files only — no code paths. Run `python3 .trellis/scripts/get_context.py --mode packages` to list available specs. Delete this line once real entries are added."}
@@ -0,0 +1,22 @@
# 本地构建控制台实施计划
1. Create a dependency-free console server and static UI shell; add `npm run console`, localhost-only binding, automatic browser launch and environment/readiness endpoint.
2. Extract/reuse small pipeline-facing helpers for area discovery, diagnostics summary, native ZIP inspection and supported tree-style options. Do not import `build-area.js` as a module.
3. Implement history cards and onboarding: config scanning, manifest/status summary, input descriptions, package/preview links and model option selection.
4. Implement safe import and Save As: upload/copy OSM+ZIP into ignored managed storage, inspect ZIP contract/areaId, atomically write a config with collision-safe output overrides, and record ownership.
5. Implement serialized build jobs with child-process log capture, SSE progress, completion/failure states, and links to the existing generated preview.
6. Implement cleanup planning and confirmation. Validate canonical paths, move only allowed targets to macOS Trash, and cover single/batch clear-output versus remove-area paths.
7. Add focused Node HTTP/unit tests, update README with launch/use/troubleshooting guidance, then manually run the actual imported-area build and preview flow.
## Validation commands
- `node --check` for each new Node entry/helper.
- focused console test command added to `package.json`.
- existing `npm run test:native-road-package`, `npm run test:build-stages`, `npm run test:preview-assets`, and `npm run test:package-contract`.
- `npm run console`, import a valid OSM/ZIP fixture, run a full build, open preview, and exercise both cleanup scopes.
## Risk controls and rollback
- Never bind beyond loopback; do not serve arbitrary paths or allow parallel builds.
- Validate cleanup targets after confirmation and immediately before Trash movement; no `rm -rf` implementation.
- If the UI path fails, existing CLI remains the supported fallback. Removing console files/script rolls back the feature without changing existing area data or output contracts.
@@ -0,0 +1,45 @@
# 本地构建控制台
## Goal
提供一个在本机 localhost 运行的图形控制台,使用户无需记忆命令行即可选择或导入道路构建输入、执行现有构建管线、查看进度并打开 Cesium 预览。
## Confirmed facts
- 当前唯一的交互入口是 `scripts/interactive-build.js`:它枚举 `config/areas/*.json`、选择阶段后调用 `scripts/build-area.js`。
- `build-area.js` 是构建、Blender/Cesium 子进程、package 发布和 preview 写入的唯一所有者;控制台必须调用它,不能复制阶段实现。
- native 区域配置通过 `nativeRoadPackage` 引用 `native-road-package/v1.1` ZIP;导入器会验证 ZIP 契约和 `areaId`,并将其安全解压到私有 pipeline cache。
- Blender 与本地文件系统不能由纯静态浏览器页面直接访问,因此入口必须由本机服务配合浏览器 UI 组成。
- 现有配置及其 `outputs/<area>/_pipeline/stages/*.manifest.json` 是历史输入和构建历史的可读取事实源,不能另建会漂移的重复数据库。
## Requirements
- R1: 以 localhost 页面作为首版交付方式;命令行入口继续可用。
- R2: 首页自动枚举并导入 `config/areas/*.json`,以区域卡片呈现可复用配置、输入摘要、最近构建状态、预览与发布包入口。
- R3: 用户可从界面选择 OSM 与 native-road ZIP,读取 ZIP 的 `areaId` 和契约信息,并在构建前给出 OSM/ZIP/区域 ID 的明确校验反馈。
- R3.1: 成功导入的 OSM 与道路 ZIP 复制到项目托管输入目录;新配置引用托管副本,保证可复现且不依赖 `/tmp` 或用户外部路径。
- R4: 保持首版简单:构建前仅可选择项目已有的植被/树木风格;Cesium 预览中可选择车辆模型。两者分别影响静态场景与演示运行时。
- R5: 用户可选构建阶段并启动现有 build-area 管线,看到实时阶段状态、日志、产物链接与可行动的失败说明。
- R6: 对历史配置的修改默认另存为新配置;覆盖已有配置必须是显式确认操作。
- R6.1: 控制台提供单项清理与批量清理,仅处理它创建的托管输入和本项目生成的区域输出;绝不删除用户在项目外选择的原始文件。首版清理统一移入系统废纸篓,保留恢复机会。
- R6.2: 清理范围分为两级,且单项、批量均适用:`清理构建产物` 仅移除区域输出并保留配置和托管输入;`移除区域` 将区域配置、控制台托管输入和区域输出一并移入废纸篓。批量操作必须先展示受影响区域和文件数,再允许确认。
- R7: 产品内置首次使用引导:说明 OSM、道路 ZIP、模型选项、构建阶段、预览入口和 Blender/Node 环境检查,不以外部 README 代替。
## Out of scope
- 改写道路编译器、ZIP 契约、Blender 几何或 Cesium 预览运行时。
- 远程部署、多用户协作、云端构建或将本机文件上传到第三方服务。
- 第一阶段打包为 macOS `.app`;待 localhost 工作流验证后再评估。
## Acceptance criteria
- [ ] 用户可在浏览器中启动、选择历史区域并重新执行一次已有构建,无需手写 CLI 参数。
- [ ] 用户可导入 OSM 与有效道路 ZIP,且在启动 Blender 前看到 ZIP contract / areaId / 文件缺失的校验结果。
- [ ] 用户可保存为新的区域配置而不改变原配置;显式覆盖有确认保护。
- [ ] 页面可展示构建阶段、实时日志、成功后的 preview URL 与 `package/manifest.json`。
- [ ] 首次使用者可仅依照页面引导完成一次输入、构建和预览,并理解失败时下一步怎么做。
## Technical notes
- native-road ZIP 的 `areaId` 与区域配置 `id` 必须相等;控制台应从 ZIP 读取并锁定该 ID,避免提供必然失败的可编辑字段。
- 控制台管理的输入需要有可判定 ownership 的目录/记录,才能安全实施区域级清理;历史配置引用的外部路径始终视为只读、非托管输入。
@@ -0,0 +1,26 @@
{
"id": "local-build-console",
"name": "local-build-console",
"title": "本地构建控制台",
"description": "",
"status": "completed",
"dev_type": null,
"scope": null,
"package": null,
"priority": "P2",
"creator": "dingkang",
"assignee": "dingkang",
"createdAt": "2026-09-17",
"completedAt": "2026-09-22",
"branch": null,
"base_branch": "experiment/road-compiler-rethink",
"worktree_path": null,
"commit": null,
"pr_url": null,
"subtasks": [],
"children": [],
"parent": null,
"relatedFiles": [],
"notes": "",
"meta": {}
}
+6 -3
View File
@@ -8,8 +8,8 @@
<!-- @@@auto:current-status -->
- **Active File**: `journal-1.md`
- **Total Sessions**: 44
- **Last Active**: 2026-08-26
- **Total Sessions**: 47
- **Last Active**: 2026-09-22
<!-- @@@/auto:current-status -->
---
@@ -19,7 +19,7 @@
<!-- @@@auto:active-documents -->
| File | Lines | Status |
|------|-------|--------|
| `journal-1.md` | ~955 | Active |
| `journal-1.md` | ~1022 | Active |
<!-- @@@/auto:active-documents -->
---
@@ -29,6 +29,9 @@
<!-- @@@auto:session-history -->
| # | Date | Title | Commits | Branch |
|---|------|-------|---------|--------|
| 47 | 2026-09-22 | 完成本地构建控制台 | `3760650` | `experiment/road-compiler-rethink` |
| 46 | 2026-09-17 | Cesium 导出性能优化与一致性验证 | `c6cce44` | `experiment/road-compiler-rethink` |
| 45 | 2026-08-26 | Complete manifest-driven native road rendering | `6201270` | `experiment/road-compiler-rethink` |
| 44 | 2026-08-26 | Complete external road compiler split | `5cde080`, `7d4b3b5` | `experiment/road-compiler-rethink` |
| 43 | 2026-08-25 | Extract road compiler package boundary | `73707da`, `d5c58b5`, `f542a64`, `284d3db`, `2469cfd` | `experiment/road-compiler-rethink` |
| 42 | 2026-08-25 | Freeze native road parity baseline | `e7bc7f8` | `experiment/road-compiler-rethink` |
+67
View File
@@ -953,3 +953,70 @@ Split road compiler into private road-compiler repository with history, pushed v
### Status
[OK] **Completed**
## Session 45: Complete manifest-driven native road rendering
**Date**: 2026-08-26
**Task**: Complete manifest-driven native road rendering
**Branch**: `experiment/road-compiler-rethink`
### Summary
Published compiler v0.2.2, switched Blender to validated native-road manifests, proved debug-probe extensibility, and verified Blender parity for fengshu-er-road.
### Git Commits
| Hash | Message |
|------|---------|
| `6201270` | (see git log) |
### Status
[OK] **Completed**
## Session 46: Cesium 导出性能优化与一致性验证
**Date**: 2026-09-17
**Task**: Cesium 导出性能优化与一致性验证
**Branch**: `experiment/road-compiler-rethink`
### Summary
基于 Blender 依赖图机制批量三角化并隔离 UV 操作;南太子湖 173.2→15.9 秒,枫树二路 36.9–41.4→5.6 秒。保留全部资产;任务已归档。
### Git Commits
| Hash | Message |
|------|---------|
| `c6cce44` | (see git log) |
### Testing
- [OK] 64 项 Python 测试、真实 Blender 特殊网格与异常清理测试通过;两区域 GLB 结构一致,南太子湖逐属性零差异;枫树树冠切线 0.0001 波动由未改代码 control 复现。
### Status
[OK] **Completed**
## Session 47: 完成本地构建控制台
**Date**: 2026-09-22
**Task**: 完成本地构建控制台
**Branch**: `experiment/road-compiler-rethink`
### Summary
提交本地构建控制台及预览逻辑改动,归档 09-17-local-build-console;通过 preview-assets 与 native-preview-traffic 测试。
### Git Commits
| Hash | Message |
|------|---------|
| `3760650` | (see git log) |
### Status
[OK] **Completed**
+138 -11
View File
@@ -496,6 +496,14 @@ def make_export_material(material):
return result
def select_only(obj):
"""Change selection without an operator's scene-evaluation barriers."""
for selected in bpy.context.selected_objects:
selected.select_set(False)
obj.select_set(True)
bpy.context.view_layer.objects.active = obj
def unwrap_mesh(obj):
if obj.type != "MESH" or not obj.data.polygons:
return
@@ -504,23 +512,19 @@ def unwrap_mesh(obj):
# meshes arrive without a UV layer, so that is the reliable discriminator.
if obj.data.uv_layers:
return
bpy.ops.object.select_all(action="DESELECT")
obj.select_set(True)
bpy.context.view_layer.objects.active = obj
select_only(obj)
bpy.ops.object.mode_set(mode="EDIT")
bpy.ops.mesh.select_all(action="SELECT")
try:
bpy.ops.mesh.select_all(action="SELECT")
bpy.ops.uv.smart_project(island_margin=0.025, area_weight=0.0)
finally:
bpy.ops.object.mode_set(mode="OBJECT")
def apply_mesh_modifiers(obj):
if obj.type != "MESH":
if obj.type != "MESH" or not obj.modifiers:
return
bpy.ops.object.select_all(action="DESELECT")
obj.select_set(True)
bpy.context.view_layer.objects.active = obj
select_only(obj)
for modifier in list(obj.modifiers):
try:
bpy.ops.object.modifier_apply(modifier=modifier.name)
@@ -528,6 +532,11 @@ def apply_mesh_modifiers(obj):
pass
def mesh_has_multiple_users(mesh):
# modifier_apply uses ID_REAL_USERS, excluding the fake retention user.
return mesh.users - int(mesh.use_fake_user) > 1
def triangulate_mesh(obj):
"""Split n-gons into triangles ahead of the exporter.
@@ -544,11 +553,14 @@ def triangulate_mesh(obj):
"""
if obj.type != "MESH" or not obj.data.polygons:
return
# modifier_apply rejects multi-user data before evaluating geometry. Avoid
# creating a modifier and evaluating the whole scene only to remove it in
# the failure handler below; keeping this mesh unchanged is intentional.
if mesh_has_multiple_users(obj.data):
return
if all(len(polygon.vertices) <= 3 for polygon in obj.data.polygons):
return
bpy.ops.object.select_all(action="DESELECT")
obj.select_set(True)
bpy.context.view_layer.objects.active = obj
select_only(obj)
modifier = obj.modifiers.new("ExportTriangulate", "TRIANGULATE")
modifier.min_vertices = 4
try:
@@ -559,6 +571,120 @@ def triangulate_mesh(obj):
obj.modifiers.remove(modifier)
def batch_triangulate_meshes(candidates):
"""Evaluate independent Triangulate modifiers in one dependency graph pass.
This uses the actual modifier (including the running Blender version's
defaults), not a BMesh reimplementation. new_from_object on an evaluated
object copies its final geometry; preserve_all_data_layers retains UVs and
other mesh attributes. Stage every result before editing original objects:
assigning data or removing a modifier invalidates evaluated state.
Existing stacks can depend on other objects' unmodified geometry. Leave
their sequential application order untouched, including stacks outside the
export subset. Multi-user meshes, shape keys and custom normals retain the
original operator path, whose restrictions are part of the export behavior.
"""
if any(obj.modifiers for obj in bpy.context.scene.objects):
return
pending = []
for obj in candidates:
mesh = obj.data
if (mesh.users != 1 or mesh.use_fake_user or mesh.shape_keys
or mesh.has_custom_normals or mesh.library or obj.library
or not any(len(p.vertices) > 3 for p in mesh.polygons)):
continue
modifier = obj.modifiers.new("ExportTriangulate", "TRIANGULATE")
modifier.min_vertices = 4
pending.append((obj, mesh, modifier))
if not pending:
return
staged = []
try:
# evaluated_get itself does not update the graph. This explicit barrier
# is required after adding all modifiers, and is intentionally not
# repeated for each object.
depsgraph = bpy.context.evaluated_depsgraph_get()
depsgraph.update()
for obj, source, modifier in pending:
mesh = bpy.data.meshes.new_from_object(
obj.evaluated_get(depsgraph), preserve_all_data_layers=True,
depsgraph=depsgraph)
staged.append((obj, source, modifier, mesh))
for key in source.keys():
mesh[key] = source[key]
except Exception:
for _, _, _, mesh in staged:
bpy.data.meshes.remove(mesh)
for obj, _, modifier in pending:
obj.modifiers.remove(modifier)
raise
for obj, source, modifier, mesh in staged:
name = source.name
obj.data = mesh
obj.modifiers.remove(modifier)
bpy.data.meshes.remove(source)
mesh.name = name
def unwrap_meshes_in_work_scene(candidates):
"""Run the unchanged per-object UV operator against a small scene.
UV projection operates on the original mesh in edit mode, not evaluated
scene geometry. Link the original object into a temporary scene one at a
time so operator barriers do not reevaluate every unrelated scene object.
Switch the window's scene, not just context object overrides: edit-mode
operators require a consistent window/scene/view-layer context.
"""
window = bpy.context.window
if window is None or any(obj.modifiers for obj in bpy.context.scene.objects):
return
# A later fallback triangulation must precede UV projection. Shared meshes
# are also safe here: modifier_apply refuses them, so their topology stays
# unchanged in the original preparation loop.
pending = [obj for obj in candidates if not obj.modifiers
and not obj.data.uv_layers and obj.data.polygons
and (mesh_has_multiple_users(obj.data)
or all(len(p.vertices) <= 3 for p in obj.data.polygons))]
if not pending:
return
original_scene = window.scene
original_layer = window.view_layer
work_scene = bpy.data.scenes.new("Cesium UV work scene")
try:
window.scene = work_scene
for obj in pending:
# Shared mesh UVs are already prepared by their first instance.
if obj.data.uv_layers:
continue
work_scene.collection.objects.link(obj)
try:
unwrap_mesh(obj)
finally:
work_scene.collection.objects.unlink(obj)
finally:
window.scene = original_scene
window.view_layer = original_layer
bpy.data.scenes.remove(work_scene)
def prepare_independent_meshes(candidates):
"""Keep dependency-bearing modifier stacks on the sequential legacy path."""
# Bevel reads this object's geometry and vertex groups. Unlike Boolean,
# Shrinkwrap or Geometry Nodes it cannot read another object's mesh. Apply
# it with the original operator before batching independent triangulation.
# Unknown stack types deliberately keep the original preparation order.
if any(modifier.type != "BEVEL"
for obj in bpy.context.scene.objects for modifier in obj.modifiers):
return
for obj in candidates:
apply_mesh_modifiers(obj)
batch_triangulate_meshes(candidates)
unwrap_meshes_in_work_scene(candidates)
def export_glb(filepath, meshes):
"""Export a prepared object subset without changing authoring visibility."""
bpy.ops.object.select_all(action="DESELECT")
@@ -654,6 +780,7 @@ def export(args):
and not obj.hide_viewport and not obj.hide_render]
total_candidates = max(1, len(candidates))
progress.update("prepare scene", 3, "%d mesh objects" % len(candidates))
prepare_independent_meshes(candidates)
for index, obj in enumerate(candidates, 1):
if any(c.name == "06_TrafficSignalsDynamic" for c in obj.users_collection):
groups = {slot.material.name for slot in obj.material_slots if slot.material}
+3 -1
View File
@@ -668,6 +668,8 @@ def build(args):
catalog.MATERIALS["native_center_line_white"])
road_mats["native_lane_separator_yellow"] = material_from_spec(
catalog.MATERIALS["native_lane_separator_yellow"])
road_mats["native_island_green"] = material_from_spec(
catalog.MATERIALS["native_island_green"])
traffic_signal_mats = {
"metal": material_from_spec(catalog.MATERIALS["traffic_signal_metal"]),
"housing": material_from_spec(catalog.MATERIALS["traffic_signal_housing"]),
@@ -800,7 +802,7 @@ def build(args):
if not os.path.isdir(native_road_dir):
raise RuntimeError("--native-road directory does not exist: " + native_road_dir)
material_layers = {layer["id"]: road_mats.get(layer["id"]) for layer in catalog.ROAD_LAYERS}
material_layers.update({"native_center_line_white": road_mats["native_center_line_white"], "native_lane_separator_yellow": road_mats["native_lane_separator_yellow"]})
material_layers.update({"native_center_line_white": road_mats["native_center_line_white"], "native_lane_separator_yellow": road_mats["native_lane_separator_yellow"], "native_island_green": road_mats["native_island_green"]})
road_counts.update(_native_roads.assemble(native_road_dir, projector, roads_c, material_layers))
elif geojson_dir and os.path.isdir(geojson_dir):
for problem in catalog.check_layers(geojson_dir):
+5
View File
@@ -194,6 +194,11 @@ MATERIALS = {
"native_lane_separator_yellow": {"kind": "solid", "name": "Native Lane Separator Yellow",
"color": (0.94, 0.58, 0.06), "roughness": 0.52,
"cesium": {"base_color": (0.94, 0.58, 0.06)}},
# Safety-island planting follows the neutral green-grey map fill and is
# intentionally opaque so it does not blend with the asphalt below.
"native_island_green": {"kind": "solid", "name": "Native Island Green",
"color": (0.0382, 0.4793, 0.1441), "roughness": 0.92,
"cesium": {"base_color": (0.0382, 0.4793, 0.1441)}},
}
+5 -1
View File
@@ -44,8 +44,12 @@ def assemble(native_road_dir, projector, collection, material_layers):
case_material_name = case.get("material", material_name)
case_material = _material(material_layers, case_material_name)
suffix = "_" + str(case["match"]) if "match" in case else ""
# Split cases may be inset overlays (for example the green
# planting inside a grey safety-island base). Keep the geometry
# unchanged, but lift matched overlays slightly to avoid z-fighting.
case_z = z_by_material_layer[material_name] + (0.001 if "match" in case else 0.0)
count += roads.assemble_geojson_layer(
source_path, source + suffix, projector, collection, case_material,
z_by_material_layer[material_name], _split_filter(split, case))
case_z, _split_filter(split, case))
counts[source] = count
return counts
+15 -1
View File
@@ -31,7 +31,7 @@ from osmassets.geom import (
sample_tree_row,
signed_polygon_area,
)
from osmassets.catalog import ROAD_LAYERS
from osmassets.catalog import MATERIALS, ROAD_LAYERS
from osmassets.native_road_manifest import CONTRACT, load as load_native_road_manifest
from osmassets.osm import Projector, parse_height, parse_osm, tags
@@ -40,6 +40,13 @@ SQUARE = [(0.0, 0.0), (10.0, 0.0), (10.0, 10.0), (0.0, 10.0)]
class RoadLayerCatalogTest(unittest.TestCase):
def test_native_island_material_is_registered(self):
material = MATERIALS["native_island_green"]
self.assertEqual(material["kind"], "solid")
self.assertEqual(material["color"], (0.0382, 0.4793, 0.1441))
self.assertGreater(material["color"][1], material["color"][0])
self.assertGreater(material["color"][1], material["color"][2])
def test_intersection_and_road_asphalt_share_the_same_colour(self):
layers = {layer["id"]: layer for layer in ROAD_LAYERS}
self.assertEqual(layers["road_surface"]["color"],
@@ -105,6 +112,13 @@ class NativeRoadManifestTest(unittest.TestCase):
with self.assertRaisesRegex(RuntimeError, "semantic"):
load_native_road_manifest(self.root.name)
def test_split_overlay_has_a_separate_z_layer(self):
native_roads = os.path.join(os.path.dirname(os.path.abspath(__file__)),
"..", "osmassets", "native_roads.py")
with open(native_roads, encoding="utf-8") as handle:
source = handle.read()
self.assertIn('0.001 if "match" in case else 0.0', source)
self.write_manifest([{"source": "center_lines", "role": "marking",
"materialLayer": "center_lines", "splitBy": {
"prop": "color", "cases": [
+177
View File
@@ -0,0 +1,177 @@
"""Compare exporter preparation against a supplied original in real Blender.
blender --background --factory-startup --python this_file.py -- \
--baseline /absolute/path/to/original_export_cesium.py
The fixture combines asymmetric quads, n-gons, authored UVs, shared meshes,
custom normals, shape keys and Bevel. A separate Boolean scenario exercises
the dependency-bearing fallback. Expected triangle totals follow n - 2;
attribute/ordering comparisons use the original exporter as an independent
reference, including an unchanged-code control run.
"""
import argparse
import importlib.util
import math
from pathlib import Path
import sys
import bpy
ROOT = Path(__file__).resolve().parents[1]
sys.path.insert(0, str(ROOT))
def load_exporter(name, path):
spec = importlib.util.spec_from_file_location(name, path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
def fixture(dependent):
bpy.ops.wm.read_factory_settings(use_empty=True)
material = bpy.data.materials.new("Fixture material")
objects = []
def mesh_object(name, vertices, faces):
mesh = bpy.data.meshes.new(name + " mesh")
mesh.from_pydata(vertices, [], faces)
mesh.materials.append(material)
mesh["fixture_property"] = "preserved"
obj = bpy.data.objects.new(name, mesh)
bpy.context.scene.collection.objects.link(obj)
objects.append(obj)
return obj
quad = [(0, 0, 0), (3, 0, 0), (2, 1, 0), (0, 2, 0)]
mesh_object("quad", quad, [(0, 1, 2, 3)])
mesh_object("ngon", quad + [(-1, 1, 0)], [(0, 1, 2, 3, 4)])
authored = mesh_object("authored", quad, [(0, 1, 2, 3)])
uv = authored.data.uv_layers.new(name="Authored UV")
for loop, value in zip(uv.data, [(0, 0), (.9, .1), (.6, .8), (.1, 1)]):
loop.uv = value
shared = mesh_object("shared", quad, [(0, 1, 2, 3)])
instance = bpy.data.objects.new("shared instance", shared.data)
bpy.context.scene.collection.objects.link(instance)
objects.append(instance)
mesh_object("triangle", quad[:3], [(0, 1, 2)])
custom = mesh_object("custom normals", quad, [(0, 1, 2, 3)])
custom.data.normals_split_custom_set([(0, .2, 1)] * 4)
keyed = mesh_object("shape keys", quad, [(0, 1, 2, 3)])
keyed.shape_key_add(name="Basis")
keyed.shape_key_add(name="Raised").data[0].co.z = .25
bevel = mesh_object("bevel", quad, [(0, 1, 2, 3)])
modifier = bevel.modifiers.new("Fixture bevel", "BEVEL")
modifier.width = .16
modifier.segments = 2
retained = mesh_object("fake user", quad, [(0, 1, 2, 3)])
retained.data.use_fake_user = True
if dependent:
boolean = objects[0].modifiers.new("Dependent Boolean", "BOOLEAN")
boolean.object = authored
return objects
def digest(objects):
return [{
"name": obj.name,
"mesh": obj.data.name,
"users": obj.data.users,
"properties": dict(obj.data.items()),
"vertices": [tuple(v.co) for v in obj.data.vertices],
"polygons": [tuple(p.vertices) for p in obj.data.polygons],
"smooth": [p.use_smooth for p in obj.data.polygons],
"normals": [tuple(n.vector) for n in obj.data.corner_normals],
"uv": {uv.name: [tuple(v.uv) for v in uv.data]
for uv in obj.data.uv_layers},
"materials": [m.name if m else None for m in obj.data.materials],
"modifiers": [(m.name, m.type) for m in obj.modifiers],
"shape_keys": ([key.name for key in obj.data.shape_keys.key_blocks]
if obj.data.shape_keys else []),
} for obj in objects]
def prepare(module, dependent, optimized):
objects = fixture(dependent)
if optimized:
module.prepare_independent_meshes(objects)
seen = set()
for obj in objects:
module.apply_mesh_modifiers(obj)
if obj.data.name not in seen:
seen.add(obj.data.name)
module.triangulate_mesh(obj)
module.unwrap_mesh(obj)
assert objects[3].data is objects[4].data, "shared mesh was split"
if not dependent:
assert len(objects[0].data.polygons) == 2, "quad must give n - 2 triangles"
assert len(objects[1].data.polygons) == 3, "pentagon must give n - 2 triangles"
return digest(objects)
def compare(expected, actual, path="root"):
if isinstance(expected, dict):
assert expected.keys() == actual.keys(), path
for key in expected:
compare(expected[key], actual[key], path + "." + str(key))
elif isinstance(expected, (list, tuple)):
assert len(expected) == len(actual), (path, len(expected), len(actual))
for index, (left, right) in enumerate(zip(expected, actual)):
compare(left, right, path + "[%d]" % index)
elif isinstance(expected, float):
assert math.isclose(expected, actual, rel_tol=0, abs_tol=1e-6), (
path, expected, actual)
else:
assert expected == actual, (path, expected, actual)
def check_work_scene_cleanup(module):
objects = fixture(False)
triangle = objects[5]
original_scene = bpy.context.window.scene
original_layer = bpy.context.window.view_layer
scene_names = set(bpy.data.scenes.keys())
collections = tuple(triangle.users_collection)
unwrap = module.unwrap_mesh
def fail_unwrap(obj):
assert bpy.context.scene != original_scene
assert obj == triangle
raise RuntimeError("fixture UV failure")
module.unwrap_mesh = fail_unwrap
# Remove the Bevel so this specifically exercises the scratch-scene path.
objects[8].modifiers.clear()
try:
try:
module.unwrap_meshes_in_work_scene([triangle])
except RuntimeError as error:
assert str(error) == "fixture UV failure"
else:
raise AssertionError("fixture failure was swallowed")
finally:
module.unwrap_mesh = unwrap
assert bpy.context.window.scene == original_scene
assert bpy.context.window.view_layer == original_layer
assert set(bpy.data.scenes.keys()) == scene_names
assert tuple(triangle.users_collection) == collections
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--baseline", required=True)
args = parser.parse_args(sys.argv[sys.argv.index("--") + 1:])
original = load_exporter("original_exporter", args.baseline)
optimized = load_exporter("optimized_exporter", ROOT / "export_cesium.py")
for dependent in (False, True):
reference = prepare(original, dependent, False)
compare(reference, prepare(original, dependent, False), "control")
compare(reference, prepare(optimized, dependent, True), "optimized")
check_work_scene_cleanup(optimized)
print("EXPORT_PREPARATION_CHECK_OK: control, independent and dependent fixtures")
if __name__ == "__main__":
main()
+1 -25
View File
@@ -1,6 +1,7 @@
{
"id": "fengshu-er-road",
"input": "/Users/que01/osm2streets-qgis-workflow/inputs/osm/枫树二路.osm",
"nativeRoadPackage": "../../inputs/native-road-packages/fengshu-er-road.zip",
"outputRoot": "/Users/que01/osm2streets-qgis-workflow/outputs",
"qgisApp": "/Applications/QGIS.app",
"blenderApp": "/Applications/Blender.app",
@@ -24,31 +25,6 @@
"turnLaneArrows": {
"enabled": false
},
"nativeRoad": {
"edgeLines": false,
"junctionTemplates": {
"enabled": true,
"references": [],
"clusters": [
{
"id": "zhushanhu-fengshu-complex",
"template": "complex-junction-v1",
"referenceFile": "../../inputs/osm/珠山湖大道(枫树二路)口.geojson",
"approachWidthMultiplier": 1.45,
"approachLengthMeters": 32,
"coreRadiusMeters": 28,
"cornerRadiusMeters": 25,
"outerRadiusExtraMeters": 24,
"nodeIds": [
"8005332807",
"8024512135",
"8024512145",
"8024512147"
]
}
]
}
},
"osm2streets": {
"debug_each_step": false,
"dual_carriageway_experiment": false,
-32
View File
@@ -1,32 +0,0 @@
{
"id": "hanyang-block",
"input": "/Users/que01/Desktop/汉阳区区块.osm",
"outputRoot": "/Users/que01/osm2streets-qgis-workflow/outputs",
"qgisApp": "/Applications/QGIS.app",
"blenderApp": "/Applications/Blender.app",
"stages": {
"intermediates": false,
"blender": true,
"cesium": true
},
"qgis": {
"arrowScale": 0.8,
"clipPad": 0.002,
"canvasPad": 0.001,
"previewPad": 0.0007,
"canvasExtent": null,
"previewExtent": null,
"layerPrefix": "osm2streets"
},
"osm2streets": {
"debug_each_step": false,
"dual_carriageway_experiment": false,
"sidepath_zipping_experiment": false,
"inferred_sidewalks": true,
"osm2lanes": true
},
"blender": {
"treeStyle": "natural",
"officeOverrides": ""
}
}
@@ -1,6 +1,7 @@
{
"id": "nantaizi-lake-innovation-valley",
"input": "/Users/que01/Desktop/南台子湖创新谷OSM.osm",
"nativeRoadPackage": "../../inputs/native-road-packages/nantaizi-lake-innovation-valley.zip",
"outputRoot": "/Users/que01/osm2streets-qgis-workflow/outputs",
"qgisApp": "/Applications/QGIS.app",
"blenderApp": "/Applications/Blender.app",
@@ -24,9 +25,6 @@
"turnLaneArrows": {
"enabled": true
},
"nativeRoad": {
"edgeLines": false
},
"osm2streets": {
"debug_each_step": false,
"dual_carriageway_experiment": false,
+16
View File
@@ -0,0 +1,16 @@
{
"id": "osm-g5Yjf1",
"input": "/Users/que01/Desktop/南台子湖创新谷OSM.osm",
"nativeRoadPackage": "/tmp/osm-g5Yjf1.native-road.zip",
"outputRoot": "/Users/que01/osm2streets-qgis-workflow/outputs",
"stages": {
"intermediates": false,
"blender": true,
"cesium": true
},
"blender": {
"treeStyle": "natural",
"officeOverrides": "",
"roadProvider": "native"
}
}
+1 -8
View File
@@ -1,6 +1,7 @@
{
"id": "my-area",
"input": "/absolute/path/to/input.osm",
"nativeRoadPackage": "/absolute/path/to/exported-native-road-package.zip",
"outputRoot": "/absolute/path/to/outputs",
"qgisApp": "/Applications/QGIS.app",
"blenderApp": "/Applications/Blender.app",
@@ -24,14 +25,6 @@
"turnLaneArrows": {
"enabled": false
},
"nativeRoad": {
"edgeLines": false,
"junctionTemplates": {
"enabled": false,
"references": [],
"clusters": []
}
},
"osm2streets": {
"debug_each_step": false,
"dual_carriageway_experiment": false,
Binary file not shown.
+1
View File
@@ -10,6 +10,7 @@
"dependencies": {
"@inquirer/prompts": "^8.5.2",
"@osm-asset/road-compiler": "git+https://git.app.que01.top/que01/road-compiler.git#v0.2.2",
"fflate": "0.8.3",
"ol": "^10.10.0",
"osm2streets-js-node": "0.1.4"
}
+3 -5
View File
@@ -5,20 +5,17 @@
"type": "commonjs",
"scripts": {
"build": "node scripts/interactive-build.js",
"console": "node scripts/build-console.js",
"build:area": "node scripts/build-area.js",
"build:qgis": "node scripts/build-osm2streets-qgis.js",
"check:area": "node scripts/check-area.js",
"compress:glb": "node scripts/compress-glb.js",
"diagnose:area": "node scripts/diagnose-area.js",
"preflight:area": "node scripts/preflight-area.js",
"road:compile": "node scripts/compile-native-roads.js",
"reference:junction": "node scripts/inspect-junction-reference.js",
"road:check": "node scripts/check-native-roads.js",
"road:workbench": "node scripts/road-workbench.js",
"serve:v2x-preview": "node scripts/v2x-preview-server.js",
"test:road-workbench": "node scripts/test-road-workbench.js",
"test:native-road": "node scripts/test-native-road.js",
"test:road-compiler-cli": "node scripts/test-road-compiler-cli.js",
"test:native-road-package": "node scripts/test-native-road-package.js",
"test:road-parity": "node scripts/test-road-parity.js",
"test:gaode-junction-reference": "node scripts/test-gaode-junction-reference.js",
"test:preflight": "node scripts/test-area-preflight.js",
@@ -38,6 +35,7 @@
"dependencies": {
"@inquirer/prompts": "^8.5.2",
"@osm-asset/road-compiler": "git+https://git.app.que01.top/que01/road-compiler.git#v0.2.2",
"fflate": "0.8.3",
"ol": "^10.10.0",
"osm2streets-js-node": "0.1.4"
}
+24 -27
View File
@@ -4,7 +4,7 @@ const fs = require("fs");
const path = require("path");
const { spawnSync } = require("child_process");
const { readAreaConfig } = require("./lib/area-config");
const { compileArea: compileNativeRoads } = require("./lib/road-compiler-cli");
const { importNativeRoadPackage } = require("./lib/native-road-package");
const { resolveStages } = require("./lib/build-stages");
const { validateManifest, addIntegrity } = require("./lib/package-contract");
const { blenderExecutable: resolveBlenderExecutable } = require("./lib/tool-paths");
@@ -16,6 +16,7 @@ const {
} = require("./lib/scene-layers");
const { digest: glbDigest } = require("./glb-digest");
const { buildVehicleRoute: buildPreviewVehicleRoute } = require("./lib/vehicle-route");
const { buildNativeTrafficSimulation } = require("./lib/native-preview-traffic-simulation");
const { writePreviewVehicleLibrary, writePreviewV2xVehicleLibrary } = require("./lib/vehicle-library");
const { readTrafficSignals } = require("./lib/traffic-signals");
const {
@@ -225,10 +226,7 @@ function buildBlenderScene(area, roadProvider) {
}
fs.mkdirSync(path.dirname(area.outputs.blend), { recursive: true });
fs.mkdirSync(path.dirname(area.outputs.render), { recursive: true });
if (roadProvider === "native") {
compileNativeRoads(area);
ensureNativeRoadLayers(area);
}
const nativePackage = roadProvider === "native" ? importNativeRoadPackage(area) : null;
const blenderArgs = [
"--background",
@@ -246,8 +244,8 @@ function buildBlenderScene(area, roadProvider) {
area.blender.treeStyle,
];
if (roadProvider === "native") {
blenderArgs.push("--native-road", area.outputs.nativeRoadDir);
blenderArgs.push("--traffic-signals", path.join(area.outputs.nativeRoadDir, "traffic-signals.json"));
blenderArgs.push("--native-road", nativePackage.rootDir);
blenderArgs.push("--traffic-signals", path.join(nativePackage.rootDir, "traffic-signals.json"));
} else {
blenderArgs.push("--geojson", area.outputs.geojsonDir);
}
@@ -270,7 +268,7 @@ function buildBlenderScene(area, roadProvider) {
inputs: {
config: fileRecord(configPath),
osm: fileRecord(area.input),
...(roadProvider === "native" ? nativeRoadRecords(area) : sceneGeojsonRecords(area)),
...(roadProvider === "native" ? nativeRoadRecords(nativePackage) : sceneGeojsonRecords(area)),
...(roadProvider === "osm2streets" ? {
geojsonDir: fileRecord(area.outputs.geojsonDir),
trafficSignalAssemblies: fileRecord(area.outputs.trafficSignalAssemblies),
@@ -282,7 +280,7 @@ function buildBlenderScene(area, roadProvider) {
render: fileRecord(area.outputs.render),
},
summary: {
...(roadProvider === "native" ? { nativeRoad: nativeRoadFeatureCounts(area) } : { geojson: geojsonFeatureCounts(area) }),
...(roadProvider === "native" ? { nativeRoad: nativeRoadFeatureCounts(nativePackage) } : { geojson: geojsonFeatureCounts(area) }),
roadProvider,
blendBytes: fileRecord(area.outputs.blend).bytes,
renderBytes: fileRecord(area.outputs.render).bytes,
@@ -291,17 +289,13 @@ function buildBlenderScene(area, roadProvider) {
});
}
function ensureNativeRoadLayers(area) {
ensureFile(path.join(area.outputs.nativeRoadDir, "compiled.json"), "Native road compilation");
for (const file of ["road_surface.geojson", "edge_lines.geojson", "intersection_surface.geojson", "sidewalk_surface.geojson", "lane_separators.geojson", "center_lines.geojson", "direction_arrows.geojson", "turn_arrows.geojson", "crosswalks.geojson", "vehicle_stop_lines.geojson"]) {
ensureFile(path.join(area.outputs.nativeRoadDir, "layers", file), `Native road layer ${file}`);
}
}
function nativeRoadRecords(area) {
const root = path.join(area.outputs.nativeRoadDir, "layers");
function nativeRoadRecords(nativePackage) {
const root = path.join(nativePackage.rootDir, "layers");
return {
nativeRoadCompiled: fileRecord(path.join(area.outputs.nativeRoadDir, "compiled.json")),
nativeRoadZip: nativePackage.zipRecord,
nativeRoadImport: fileRecord(nativePackage.rootDir),
nativeRoadCompiled: fileRecord(path.join(nativePackage.rootDir, "compiled.json")),
nativeRoadSignals: fileRecord(path.join(nativePackage.rootDir, "traffic-signals.json")),
nativeRoadSurface: fileRecord(path.join(root, "road_surface.geojson")),
nativeEdgeLines: fileRecord(path.join(root, "edge_lines.geojson")),
nativeIntersectionSurface: fileRecord(path.join(root, "intersection_surface.geojson")),
@@ -315,8 +309,8 @@ function nativeRoadRecords(area) {
};
}
function nativeRoadFeatureCounts(area) {
const root = path.join(area.outputs.nativeRoadDir, "layers");
function nativeRoadFeatureCounts(nativePackage) {
const root = path.join(nativePackage.rootDir, "layers");
return {
roadSurface: featureCount(path.join(root, "road_surface.geojson")),
edgeLines: featureCount(path.join(root, "edge_lines.geojson")),
@@ -487,8 +481,9 @@ function compressCesiumGlb(area) {
function publishPackage(area, roadProvider) {
ensureFile(area.outputs.packageStagingManifest, "Staged package manifest");
const trafficSignalsSource = roadProvider === "native"
? path.join(area.outputs.nativeRoadDir, "traffic-signals.json")
const nativePackage = roadProvider === "native" ? importNativeRoadPackage(area) : null;
const trafficSignalsSource = nativePackage
? path.join(nativePackage.rootDir, "traffic-signals.json")
: area.outputs.trafficSignals;
ensureFile(trafficSignalsSource, "Traffic signal anchors");
const started = Date.now();
@@ -574,10 +569,12 @@ function writeCesiumPreview(area, roadProvider) {
intersectionSurface: fileRecord(intersectionSurface),
});
} else {
Object.assign(previewInputs, nativeRoadRecords(area));
// This operational preview intentionally has no synthetic vehicle source.
// A stale descriptor from an older build must not resurrect simulated routes.
fs.rmSync(area.outputs.trafficSimulation, { force: true });
const nativePackage = importNativeRoadPackage(area);
Object.assign(previewInputs, nativeRoadRecords(nativePackage));
const simulation = buildNativeTrafficSimulation(area, { nativePackage });
fs.mkdirSync(path.dirname(area.outputs.trafficSimulation), { recursive: true });
fs.writeFileSync(area.outputs.trafficSimulation, `${JSON.stringify(simulation, null, 2)}\n`);
routeArtifact = area.outputs.trafficSimulation;
}
const htmlPath = area.outputs.cesiumPreview;
const started = Date.now();
+25
View File
@@ -0,0 +1,25 @@
#!/usr/bin/env node
"use strict";
const http = require("http"), fs = require("fs"), path = require("path"), crypto = require("crypto"), child = require("child_process");
const { unzipSync } = require("fflate");
const { readAreaConfig } = require("./lib/area-config");
const { STAGES } = require("./lib/build-stages");
const root = path.resolve(__dirname, ".."), configDir = path.join(root, "config", "areas"), managedRoot = path.join(root, "inputs", "console"), stateRoot = path.join(root, ".build-console", "areas"), uiFile = path.join(__dirname, "console", "index.html");
let active = null;
const send = (res, status, data) => { res.writeHead(status, { "content-type": "application/json; charset=utf-8", "cache-control": "no-store" }); res.end(JSON.stringify(data)); };
const safeKey = value => { const key = String(value || "").toLowerCase().replace(/[^a-z0-9_-]+/g, "-").replace(/^-+|-+$/g, ""); if (!key) throw Error("请输入只含字母、数字、- 或 _ 的配置名称"); return key; };
const under = (file, base) => { const resolved = path.resolve(file), parent = path.resolve(base); return resolved === parent || resolved.startsWith(`${parent}${path.sep}`); };
function areas() { return fs.readdirSync(configDir).filter(x => x.endsWith(".json")).sort().map(file => { const config = path.join(configDir, file); try { const a = readAreaConfig(config, { repoRoot: root }); const preview = a.outputs.cesiumPreview, manifest = a.outputs.packageManifest; return { key:path.basename(file,".json"), id:a.id, config, input:a.input, zip:a.nativeRoadPackage, treeStyle:a.blender.treeStyle, output:a.outputs.areaDir, preview:fs.existsSync(preview) ? `/artifacts/${path.basename(file,".json")}/${path.basename(preview)}` : null, manifest:fs.existsSync(manifest) ? `/artifacts/${path.basename(file,".json")}/package/manifest.json` : null }; } catch (e) { return { key:path.basename(file,".json"), error:e.message }; } }); }
function inspectZip(bytes) { let files; try { files = unzipSync(bytes); } catch (_) { throw Error("道路 ZIP 无法解压"); } if (!files["manifest.json"]) throw Error("道路 ZIP 缺少 manifest.json"); let manifest; try { manifest=JSON.parse(Buffer.from(files["manifest.json"]).toString("utf8")); } catch (_) { throw Error("道路 ZIP manifest.json 无效"); } if (manifest.contract !== "native-road-package/v1.1" || !manifest.areaId) throw Error("道路 ZIP 不是受支持的 native-road-package/v1.1"); return { areaId:manifest.areaId, contract:manifest.contract }; }
function body(req) { return new Promise((resolve,reject) => { let data=""; req.on("data", c=>{ data+=c; if(data.length>100*1024*1024) reject(Error("上传文件过大")); }); req.on("end",()=>{ try { resolve(JSON.parse(data||"{}")); } catch (_) { reject(Error("请求 JSON 无效")); } }); }); }
function trash(target) { if (!fs.existsSync(target)) return; if (!under(target, root)) throw Error("拒绝清理项目外文件"); child.execFileSync("osascript", ["-e", `tell application \"Finder\" to delete POSIX file ${JSON.stringify(target)}`]); }
function serve(req,res) { const url = new URL(req.url, "http://127.0.0.1"); if (req.method==="GET" && url.pathname==="/") { res.writeHead(200,{"content-type":"text/html; charset=utf-8"}); return res.end(fs.readFileSync(uiFile)); }
if(req.method==="GET"&&url.pathname==="/api/health") return send(res,200,{node:process.version, blender:fs.existsSync("/Applications/Blender.app/Contents/MacOS/Blender"), areas:areas().length});
if(req.method==="GET"&&url.pathname==="/api/areas") return send(res,200,{areas:areas(),stages:STAGES,treeStyles:["natural","procedural","shapespark"]});
if(req.method==="POST"&&url.pathname==="/api/inspect") return body(req).then(x=>send(res,200,inspectZip(Buffer.from(x.zip,"base64")))).catch(e=>send(res,400,{error:e.message}));
if(req.method==="POST"&&url.pathname==="/api/import") return body(req).then(x=>{ const key=safeKey(x.key), osm=Buffer.from(x.osm,"base64"), zip=Buffer.from(x.zip,"base64"), info=inspectZip(zip), dir=path.join(managedRoot,key); if(fs.existsSync(path.join(configDir,`${key}.json`))) throw Error("该配置名称已存在;请另存为新名称"); fs.mkdirSync(dir,{recursive:true}); const osmFile=path.join(dir,"source.osm"), zipFile=path.join(dir,"roads.zip"); fs.writeFileSync(osmFile,osm); fs.writeFileSync(zipFile,zip); const out=path.join(root,"outputs",key), config={id:info.areaId,input:osmFile,nativeRoadPackage:zipFile,outputRoot:path.join(root,"outputs"),outputs:{areaDir:out,fileStem:key},blender:{roadProvider:"native",treeStyle:x.treeStyle||"natural"}}; const file=path.join(configDir,`${key}.json`); fs.writeFileSync(file,JSON.stringify(config,null,2)+"\n"); fs.mkdirSync(stateRoot,{recursive:true}); fs.writeFileSync(path.join(stateRoot,`${key}.json`),JSON.stringify({key,config:file,managed:dir,output:out},null,2)); send(res,201,{key,id:info.areaId}); }).catch(e=>send(res,400,{error:e.message}));
if(req.method==="POST"&&url.pathname==="/api/build") return body(req).then(x=>{ if(active&&!active.done) throw Error("已有构建正在运行"); const key=safeKey(x.key), file=path.join(configDir,`${key}.json`); if(!fs.existsSync(file)) throw Error("配置不存在"); const args=[path.join(__dirname,"build-area.js"),"--config",file]; if(Array.isArray(x.stages)&&x.stages.length) args.push("--stages",x.stages.join(",")); const proc=child.spawn(process.execPath,args,{cwd:root}); const job=active={id:crypto.randomUUID(),key,logs:[],done:false}; for(const s of [proc.stdout,proc.stderr]) s.on("data",b=>job.logs.push(b.toString())); proc.on("close",code=>{job.logs.push(`\n构建结束:${code===0?"成功":"失败"} (${code})\n`);job.done=true;job.code=code;}); send(res,202,{id:job.id}); }).catch(e=>send(res,409,{error:e.message}));
if(req.method==="GET"&&url.pathname==="/api/job") return send(res,200,active||{done:true});
if(req.method==="POST"&&url.pathname==="/api/cleanup") return body(req).then(x=>{ const keys=Array.isArray(x.keys)?x.keys:[]; const known=new Map(areas().map(a=>[a.key,a])); for(const raw of keys){const key=safeKey(raw), area=known.get(key), config=path.join(configDir,`${key}.json`); if(!area||!fs.existsSync(config)) throw Error(`区域不存在:${key}`); trash(area.output||path.join(root,"outputs",key)); const rec=path.join(stateRoot,`${key}.json`); if(x.mode==="remove"){ if(fs.existsSync(rec)){const r=JSON.parse(fs.readFileSync(rec));trash(r.managed);trash(rec);} trash(config); } } send(res,200,{ok:true}); }).catch(e=>send(res,400,{error:e.message}));
if(req.method==="GET"&&url.pathname.startsWith("/artifacts/")){const [, ,key,...rest]=url.pathname.split("/");const a=areas().find(x=>x.key===key);const f=path.resolve(a?.output||"",...rest);if(!a||!under(f,a.output)||!fs.existsSync(f))return send(res,404,{error:"未找到产物"});return fs.createReadStream(f).pipe(res);} send(res,404,{error:"Not found"}); }
http.createServer((q,s)=>serve(q,s)).listen(8788,"127.0.0.1",()=>{const u="http://127.0.0.1:8788/";console.log(`Build console: ${u}`);child.spawn("open",[u],{detached:true,stdio:"ignore"}).unref();});
+8
View File
@@ -0,0 +1,8 @@
<!doctype html><meta charset="utf-8"><title>OSM Asset Console</title><style>:root{--ink:#172033;--muted:#718096;--line:#dde3ed;--blue:#3867e8;--bg:#f3f6fb}*{box-sizing:border-box}body{font:15px/1.6 Inter,system-ui;margin:0 auto;max-width:1180px;padding:44px 30px 80px;background:var(--bg);color:var(--ink)}h1{font-size:42px;letter-spacing:-1.8px;margin:0}h1+p{max-width:660px;color:var(--muted);margin-top:6px;font-size:16px}section{background:white;border:1px solid var(--line);border-radius:18px;padding:24px;margin:16px 0;box-shadow:0 12px 28px #13244a0a}h2{font-size:18px;margin:0 0 14px}button,input,select{font:inherit;padding:10px 12px;border:1px solid var(--line);border-radius:9px;background:#fff;margin:3px}button{background:var(--blue);color:#fff;border-color:var(--blue);font-weight:650;cursor:pointer}button:hover{filter:brightness(.92)}#health{display:inline-block;padding:9px 14px;border-radius:999px;color:var(--blue);font-weight:650;background:#edf2ff}#clear{background:#526076;border-color:#526076}#remove{background:#c33b35;border-color:#c33b35}.card{display:grid;grid-template-columns:1fr auto;border-top:1px solid #edf0f5;padding:18px 4px;gap:8px}.card:first-child{border:0}.card a{color:var(--blue);font-weight:650;margin-right:12px}.card button{grid-column:2;grid-row:1/4;align-self:center}pre{margin:0;padding:18px;border-radius:12px;background:#111827;color:#d8e0ec;max-height:340px;overflow:auto}@media(max-width:640px){body{padding:28px 16px}.card{display:block}}</style><h1>OSM Asset Console</h1><p>从道路包到可交付场景的本地工作台。导入输入、构建资产、检查预览,全程保留可复现的项目记录。</p><section id="health">正在检查环境…</section><section><h2>新建区域</h2><input id="key" placeholder="配置名称,例如 lake-v2"><input id="osm" type="file" accept=".osm,.xml"><input id="zip" type="file" accept=".zip"><select id="tree"><option>natural</option><option>procedural</option><option>shapespark</option></select><button id="inspect">检查 ZIP</button><button id="import">导入并保存</button><span id="note"></span></section><section><h2>历史区域</h2><div id="areas"></div><button id="clear">清理选中构建产物</button><button id="remove">移除选中区域</button><p>清理仅影响控制台托管的输入/输出,并移入废纸篓。</p></section><section><h2>构建日志</h2><pre id="log">尚未启动构建</pre></section><script>
const $=id=>document.getElementById(id), api=(u,o={})=>fetch(u,{headers:{'content-type':'application/json'},...o}).then(async r=>{let x=await r.json();if(!r.ok)throw Error(x.error);return x}), b64=f=>new Promise((ok,no)=>{let r=new FileReader;r.onload=()=>ok(r.result.split(',')[1]);r.onerror=no;r.readAsDataURL(f)});let selected=[];
async function load(){let [h,d]=await Promise.all([api('/api/health'),api('/api/areas')]);$('health').textContent=`Node ${h.node} · Blender ${h.blender?'已就绪':'未找到'} · ${h.areas} 个历史配置`; $('areas').innerHTML=d.areas.map(a=>`<div class=card><label><input type=checkbox value="${a.key}"> <b>${a.key}</b></label> — ${a.id||a.error}<br>树木:${a.treeStyle||'-'}<br>${a.preview?`<a target=_blank href="${a.preview}">打开预览</a> `:''}${a.manifest?`<a target=_blank href="${a.manifest}">package manifest</a> `:''}<button data-build="${a.key}">构建</button></div>`).join('');document.querySelectorAll('[data-build]').forEach(x=>x.onclick=()=>build(x.dataset.build));document.querySelectorAll('#areas input').forEach(x=>x.onchange=()=>selected=[...document.querySelectorAll('#areas input:checked')].map(x=>x.value));}
async function build(key){try{let r=await api('/api/build',{method:'POST',body:JSON.stringify({key})});$('note').textContent=`正在构建 ${key}…`;let t=setInterval(async()=>{let j=await api('/api/job');$('log').textContent=j.logs.join('');if(j.done){clearInterval(t);$('note').textContent=j.code===0?'构建完成,可打开预览':'构建失败,请查看日志';load()}},700)}catch(e){$('note').textContent=`无法启动构建:${e.message}`}}
$('inspect').onclick=async()=>{try{$('note').textContent='ZIP: '+JSON.stringify(await api('/api/inspect',{method:'POST',body:JSON.stringify({zip:await b64($('zip').files[0])})}))}catch(e){$('note').textContent=e.message}};
$('import').onclick=async()=>{try{let r=await api('/api/import',{method:'POST',body:JSON.stringify({key:$('key').value,osm:await b64($('osm').files[0]),zip:await b64($('zip').files[0]),treeStyle:$('tree').value})});$('note').textContent=`已保存 ${r.key}`;load()}catch(e){$('note').textContent=e.message}};
for(let [id,mode] of [['clear','outputs'],['remove','remove']])$(id).onclick=async()=>{if(!selected.length){$('note').textContent='请先勾选一个或多个区域';return}if(confirm(`将 ${selected.join(', ')} 移入废纸篓?`)){try{await api('/api/cleanup',{method:'POST',body:JSON.stringify({keys:selected,mode})});selected=[];$('note').textContent='已移入废纸篓';load()}catch(e){$('note').textContent=`清理失败:${e.message}`}}};load();
</script>
+8 -25
View File
@@ -25,6 +25,10 @@ function normalizeAreaConfig(raw, options = {}) {
const fileStem = outputOverrides.fileStem || id;
const compress = normalizeCompressConfig(raw.compress);
const budget = normalizeBudgetConfig(raw.budget);
const roadProvider = roadProviderOption(raw.blender?.roadProvider ?? "native");
if (roadProvider === "native" && raw.nativeRoadPackage === undefined) {
throw new Error("Missing config key: nativeRoadPackage");
}
const pipelineDir = path.resolve(outputOverrides.pipelineDir || path.join(areaDir, "_pipeline"));
const packageDir = path.resolve(outputOverrides.packageDir || path.join(areaDir, "package"));
const packageStagingDir = path.resolve(outputOverrides.packageStagingDir || path.join(pipelineDir, "package-staging"));
@@ -32,13 +36,9 @@ function normalizeAreaConfig(raw, options = {}) {
const packageStagingRuntimeDir = path.resolve(outputOverrides.packageStagingRuntimeDir || path.join(packageStagingDir, "runtime"));
const previewDir = path.resolve(outputOverrides.previewDir || path.join(areaDir, "_preview"));
const geojsonDir = path.resolve(outputOverrides.geojsonDir || path.join(areaDir, "osm2streets_web_out"));
const nativeRoadDir = path.resolve(outputOverrides.nativeRoadDir || path.join(areaDir, "native-road"));
const outputs = {
areaDir,
geojsonDir,
nativeRoadDir,
nativeRoadOverrides: path.resolve(outputOverrides.nativeRoadOverrides || path.join(areaDir, "native-road-overrides.json")),
nativeTrafficSignals: path.resolve(outputOverrides.nativeTrafficSignals || path.join(areaDir, "native-traffic-signals.json")),
gpkg: path.resolve(outputOverrides.gpkg || path.join(areaDir, `${fileStem}.gpkg`)),
qgisProject: path.resolve(outputOverrides.qgisProject || path.join(areaDir, `${fileStem}.qgz`)),
qgisPreview: path.resolve(outputOverrides.qgisPreview || path.join(areaDir, `${fileStem}-preview.png`)),
@@ -81,6 +81,9 @@ function normalizeAreaConfig(raw, options = {}) {
return {
id,
nativeRoadPackage: raw.nativeRoadPackage === undefined
? null
: path.resolve(options.configDir || repoRoot, requireText(raw.nativeRoadPackage, "nativeRoadPackage")),
input,
outputRoot,
qgisApp: raw.qgisApp || (process.platform === "darwin" ? "/Applications/QGIS.app" : "/usr"),
@@ -111,10 +114,6 @@ function normalizeAreaConfig(raw, options = {}) {
turnLaneArrows: {
enabled: booleanOption(raw.turnLaneArrows?.enabled, false, "turnLaneArrows.enabled"),
},
nativeRoad: {
edgeLines: booleanOption(raw.nativeRoad?.edgeLines, false, "nativeRoad.edgeLines"),
junctionTemplates: normalizeJunctionTemplates(raw.nativeRoad?.junctionTemplates, repoRoot, options.configDir || repoRoot),
},
osm2streets: raw.osm2streets || {
debug_each_step: false,
dual_carriageway_experiment: false,
@@ -125,7 +124,7 @@ function normalizeAreaConfig(raw, options = {}) {
blender: {
treeStyle: raw.blender?.treeStyle || "natural",
officeOverrides: raw.blender?.officeOverrides || raw.blender?.office_overrides || "",
roadProvider: roadProviderOption(raw.blender?.roadProvider ?? "native"),
roadProvider,
},
v2xPreview: normalizeV2xPreviewConfig(raw.v2xPreview),
compress,
@@ -272,24 +271,8 @@ function booleanOption(value, fallback, label) {
throw new Error(`${label} must be boolean`);
}
function toRoadCompilerInput(area) {
return {
areaId: area.id,
osmFile: area.input,
outDir: area.outputs.nativeRoadDir,
stagingDir: area.outputs.pipelineDir,
overridesFile: area.outputs.nativeRoadOverrides,
trafficSignalsFile: area.outputs.nativeTrafficSignals,
comparisonDir: area.outputs.geojsonDir,
options: {
edgeLines: area.nativeRoad.edgeLines,
junctionTemplates: area.nativeRoad.junctionTemplates,
},
};
}
module.exports = {
normalizeAreaConfig,
readAreaConfig,
toRoadCompilerInput,
};
+24 -11
View File
@@ -3,6 +3,7 @@
const fs = require("fs");
const path = require("path");
const { readAreaConfig } = require("./area-config");
const { importNativeRoadPackage } = require("./native-road-package");
const {
SCENE_LAYERS,
SCENE_FILE,
@@ -324,6 +325,11 @@ function addEndpoint(map, ref) {
function artifactStatus(area) {
const native = area.blender.roadProvider === "native";
let nativePackage = null;
let nativePackageError = null;
if (native) {
try { nativePackage = importNativeRoadPackage(area); } catch (error) { nativePackageError = error.message; }
}
const entries = [
...(native ? [] : [
["GeoJSON dir", area.outputs.geojsonDir, true, "dir"],
@@ -332,15 +338,16 @@ function artifactStatus(area) {
["QGIS preview", area.outputs.qgisPreview, true, "file"],
["Traffic signal assemblies", area.outputs.trafficSignalAssemblies, true, "file"],
]),
["Native road directory", area.outputs.nativeRoadDir, native, "dir"],
["Traffic signal runtime", native ? path.join(area.outputs.nativeRoadDir, "traffic-signals.json") : area.outputs.packageTrafficSignals, true, "file"],
["Native road ZIP", area.nativeRoadPackage, native, "file"],
["Native road import", nativePackage?.rootDir || path.join(area.outputs.pipelineDir, "native-road-import"), native, "dir"],
["Traffic signal runtime", native ? path.join(nativePackage?.rootDir || "", "traffic-signals.json") : area.outputs.packageTrafficSignals, true, "file"],
["Blend scene", area.outputs.blend, true, "file"],
["Render PNG", area.outputs.render, true, "file"],
["Package manifest", area.outputs.packageManifest, true, "file"],
["Package primary GLB", area.outputs.packagePrimaryGlb, true, "file"],
["Cesium preview", area.outputs.cesiumPreview, true, "file"],
];
return entries.map(([label, file, expected, type]) => {
const result = entries.map(([label, file, expected, type]) => {
const exists = fs.existsSync(file);
const stat = exists ? fs.statSync(file) : null;
const validType = !exists || (type === "dir" ? stat.isDirectory() : stat.isFile());
@@ -358,6 +365,8 @@ function artifactStatus(area) {
geojsonFiles,
};
});
if (nativePackageError) result.push({ label: "Native road ZIP validation", path: area.nativeRoadPackage, expected: true, type: "file", exists: false, validType: false, error: nativePackageError });
return result;
}
function metadataSummary(file, warnings) {
@@ -378,6 +387,10 @@ function metadataSummary(file, warnings) {
function stageManifestStatus(area, configPath = null) {
const native = area.blender.roadProvider === "native";
let nativePackage = null;
if (native) {
try { nativePackage = importNativeRoadPackage(area); } catch (_) { /* artifact status reports the validation failure */ }
}
const compressionComplete = fs.existsSync(stageManifestPath(area, "compress"));
const reimportManifest = stageManifestPath(area, "reimport");
const hasReimportManifest = fs.existsSync(reimportManifest);
@@ -435,7 +448,7 @@ function stageManifestStatus(area, configPath = null) {
inputs: {
...(configPath ? { config: configPath } : {}),
osm: area.input,
...(native ? nativeRoadRecords(area) : {
...(native && nativePackage ? nativeRoadRecords(nativePackage) : {
geojsonDir: area.outputs.geojsonDir,
...sceneGeojsonFiles(area),
trafficSignalAssemblies: area.outputs.trafficSignalAssemblies,
@@ -468,18 +481,16 @@ function stageManifestStatus(area, configPath = null) {
glb: area.outputs.glb,
metadata: area.outputs.metadata,
}),
...(native ? nativeRoadRecords(area) : {
...(native && nativePackage ? nativeRoadRecords(nativePackage) : {
lanePolygons: path.join(area.outputs.geojsonDir, "lane_polygons.geojson"),
network: path.join(area.outputs.geojsonDir, "network.json"),
intersectionSurface: path.join(area.outputs.geojsonDir, "intersection_surface.geojson"),
}),
...(native ? { trafficSimulation: area.outputs.trafficSimulation } : {}),
previewCss: path.join(path.resolve(__dirname, ".."), "lib", "cesium-preview.css"),
previewJs: path.join(path.resolve(__dirname, ".."), "lib", "cesium-preview.js"),
},
outputs: compressionComplete ? {
vehicleRoute: native ? optionalExpectedFile(area.outputs.vehicleRoute) : area.outputs.vehicleRoute,
...(native ? { trafficSimulation: area.outputs.trafficSimulation } : {}),
vehicleModel: area.outputs.vehicleModel,
} : {
cesiumPreview: area.outputs.cesiumPreview,
@@ -609,11 +620,13 @@ function sceneGeojsonFiles(area) {
return files;
}
function nativeRoadRecords(area) {
const root = path.join(area.outputs.nativeRoadDir, "layers");
function nativeRoadRecords(nativePackage) {
const root = path.join(nativePackage.rootDir, "layers");
return {
nativeRoadCompiled: path.join(area.outputs.nativeRoadDir, "compiled.json"),
nativeRoadSignals: path.join(area.outputs.nativeRoadDir, "traffic-signals.json"),
nativeRoadZip: nativePackage.zipFile,
nativeRoadImport: nativePackage.rootDir,
nativeRoadCompiled: path.join(nativePackage.rootDir, "compiled.json"),
nativeRoadSignals: path.join(nativePackage.rootDir, "traffic-signals.json"),
nativeRoadSurface: path.join(root, "road_surface.geojson"),
nativeEdgeLines: path.join(root, "edge_lines.geojson"),
nativeIntersectionSurface: path.join(root, "intersection_surface.geojson"),
+9 -3
View File
@@ -49,14 +49,20 @@
const descriptor = config.previewDescriptorName ? await fetchOptionalJson(config.previewDescriptorName) : null;
adaptPackageManifest(metadata, descriptor);
const signalData = await fetchOptionalJson(config.trafficSignalsName);
const routeData = await fetchOptionalJson(config.routeName);
const placement = scenePlacement(metadata);
const viewer = createViewer();
setLoadingMessage("Loading model", config.glbName || "");
const assets = await loadSceneAssets(viewer, metadata, placement);
const trafficStart = Cesium.JulianDate.now();
const trafficSignals = createLiveTrafficSignals(signalData, assets);
hideUnconfirmedSignalAssets(assets);
const cruise = createLiveVehicleState();
const useLiveV2x = Boolean(config.v2xPreview?.enabled);
const trafficSignals = useLiveV2x
? createLiveTrafficSignals(signalData, assets)
: addTrafficSignals(viewer, signalData, trafficStart, assets);
if (useLiveV2x) hideUnconfirmedSignalAssets(assets);
const cruise = useLiveV2x
? createLiveVehicleState()
: addVehicleCruises(viewer, routeData, signalData, trafficStart, config.vehicleModelNames, config.vehicleModelName);
const cameras = createCameraPresets(viewer, metadata, placement, cruise);
const v2xOverlay = typeof window.createV2xCesiumOverlay === "function"
? window.createV2xCesiumOverlay({
@@ -19,7 +19,8 @@ const DEFAULT_SETTINGS = {
};
function buildNativeTrafficSimulation(area, options = {}) {
const root = area.outputs.nativeRoadDir;
const root = options.nativePackage?.rootDir;
if (!root) throw new Error("Native traffic simulation requires an imported native road package");
const compiledPath = path.join(root, "compiled.json");
const signalPath = path.join(root, "traffic-signals.json");
const connectorPath = path.join(root, "layers", "connectors.geojson");
+87
View File
@@ -0,0 +1,87 @@
"use strict";
const crypto = require("crypto");
const fs = require("fs");
const path = require("path");
const { unzipSync } = require("fflate");
const CONTRACT = "native-road-package/v1.1";
const ROOT_FILES = ["manifest.json", "compiled.json", "diagnostics.json", "comparison.json", "traffic-signal-assemblies.json", "traffic-signals.json"];
function sha256(file) { return crypto.createHash("sha256").update(fs.readFileSync(file)).digest("hex"); }
function readCentralEntries(bytes) {
const entries = [];
for (let i = 0; i + 46 <= bytes.length; i += 1) {
if (bytes.readUInt32LE(i) !== 0x02014b50) continue;
const nameLength = bytes.readUInt16LE(i + 28);
const extraLength = bytes.readUInt16LE(i + 30);
const commentLength = bytes.readUInt16LE(i + 32);
const name = bytes.subarray(i + 46, i + 46 + nameLength).toString("utf8");
const externalAttrs = bytes.readUInt32LE(i + 38);
entries.push({ name, externalAttrs, directory: name.endsWith("/") });
i += 45 + nameLength + extraLength + commentLength;
}
if (!entries.length) throw new Error("ZIP central directory is missing");
return entries;
}
function validateName(name) {
if (!name || path.posix.isAbsolute(name) || name.split("/").includes("..") || name.includes("\\")) throw new Error(`Unsafe ZIP entry path: ${name}`);
if (name.startsWith("./") || (name.includes("/") && !name.startsWith("layers/")) || (name.startsWith("layers/") && name.split("/").length !== 2)) throw new Error(`Native road ZIP must be root-flat: ${name}`);
}
function validateManifest(rootDir, areaId) {
let manifest;
try { manifest = JSON.parse(fs.readFileSync(path.join(rootDir, "manifest.json"), "utf8")); } catch (error) { throw new Error(`Invalid native road manifest: ${error.message}`); }
if (!manifest || manifest.contract !== CONTRACT) throw new Error(`Unsupported native road contract: ${manifest?.contract || "missing"}`);
if (manifest.areaId !== areaId) throw new Error(`Native road areaId mismatch: expected ${areaId}, got ${manifest.areaId}`);
if (!Array.isArray(manifest.layers) || !manifest.layers.length) throw new Error("Native road manifest layers are missing");
const declared = new Set();
for (const layer of manifest.layers) {
if (!layer || typeof layer.source !== "string" || layer.source !== path.posix.basename(layer.source) || declared.has(layer.source)) throw new Error("Invalid native road layer declaration");
if (!["surface", "marking", "semantic"].includes(layer.role)) throw new Error(`Invalid native road layer role: ${layer.role}`);
if (layer.role !== "semantic" && typeof layer.materialLayer !== "string") throw new Error(`Renderable layer ${layer.source} has no materialLayer`);
declared.add(layer.source);
}
const published = new Set(fs.readdirSync(path.join(rootDir, "layers")).filter((name) => name.endsWith(".geojson")).map((name) => name.slice(0, -8)));
if (declared.size !== published.size || [...declared].some((name) => !published.has(name))) throw new Error("Native road manifest/source mismatch");
return manifest;
}
function importNativeRoadPackage(area) {
const zipFile = area.nativeRoadPackage;
if (!zipFile || !fs.existsSync(zipFile) || !fs.statSync(zipFile).isFile()) throw new Error(`Native road ZIP not found: ${zipFile || "missing nativeRoadPackage"}`);
const bytes = fs.readFileSync(zipFile);
const hash = crypto.createHash("sha256").update(bytes).digest("hex");
const entries = readCentralEntries(bytes);
const names = new Set();
for (const entry of entries) {
validateName(entry.name);
if (names.has(entry.name)) throw new Error(`Duplicate ZIP entry: ${entry.name}`);
names.add(entry.name);
const unixMode = entry.externalAttrs >>> 16;
if ((unixMode & 0xf000) === 0xa000) throw new Error(`Symbolic links are not allowed: ${entry.name}`);
}
let files;
try { files = unzipSync(bytes); } catch (error) { throw new Error(`Invalid ZIP archive: ${error.message}`); }
const expected = new Set([...ROOT_FILES, ...[...names].filter((name) => name.startsWith("layers/") && name.endsWith(".geojson"))]);
for (const name of ROOT_FILES) if (!names.has(name)) throw new Error(`Missing native road ZIP entry: ${name}`);
for (const name of names) if (!expected.has(name) || (!name.startsWith("layers/") && !ROOT_FILES.includes(name))) throw new Error(`Unexpected native road ZIP entry: ${name}`);
const rootDir = path.join(area.outputs.pipelineDir, "native-road-import", hash);
if (!fs.existsSync(rootDir)) {
fs.mkdirSync(path.join(rootDir, "layers"), { recursive: true });
for (const [name, data] of Object.entries(files)) {
if (!names.has(name) || name.endsWith("/")) continue;
const destination = path.join(rootDir, name);
fs.mkdirSync(path.dirname(destination), { recursive: true });
fs.writeFileSync(destination, data);
}
}
const manifest = validateManifest(rootDir, area.id);
const records = {};
for (const name of names) { const file = path.join(rootDir, name); if (fs.existsSync(file) && fs.statSync(file).isFile()) records[name] = { path: file, bytes: fs.statSync(file).size, sha256: sha256(file) }; }
return Object.freeze({ zipFile, zipRecord: { path: zipFile, bytes: bytes.length, sha256: hash }, rootDir, manifest, records });
}
module.exports = { CONTRACT, importNativeRoadPackage };
+5 -3
View File
@@ -5,6 +5,7 @@ const crypto = require("crypto");
const fs = require("fs");
const path = require("path");
const { readAreaConfig } = require("./lib/area-config");
const { importNativeRoadPackage } = require("./lib/native-road-package");
const repoRoot = path.resolve(__dirname, "..");
@@ -77,10 +78,11 @@ function filesBelow(directory) {
function snapshot(configPath) {
const area = readAreaConfig(configPath, { repoRoot });
if (!fs.existsSync(area.outputs.nativeRoadDir)) throw new Error(`Native road output not found: ${area.outputs.nativeRoadDir}`);
const nativePackage = importNativeRoadPackage(area);
const nativeRoadDir = nativePackage.rootDir;
const files = {};
for (const file of filesBelow(area.outputs.nativeRoadDir).sort()) {
const relative = path.relative(area.outputs.nativeRoadDir, file).split(path.sep).join("/");
for (const file of filesBelow(nativeRoadDir).sort()) {
const relative = path.relative(nativeRoadDir, file).split(path.sep).join("/");
const text = fs.readFileSync(file, "utf8");
files[relative] = jsonRecord(text, Buffer.byteLength(text));
}
+4 -4
View File
@@ -19,7 +19,7 @@ assert.match(areaBuildSource, /exportCesium\(area, roadProvider\)/);
assert.match(areaBuildSource, /"--dynamic-glb", area\.outputs\.trafficSignalsDynamicGlb/);
assert.match(areaBuildSource, /trafficSignalsCountdown0Glb: fileRecord/);
assert.match(areaBuildSource, /if \(roadProvider === "osm2streets"\) \{[\s\S]*?buildPreviewVehicleRoute/);
assert.match(areaBuildSource, /Object\.assign\(previewInputs, nativeRoadRecords\(area\)\)/);
assert.match(areaBuildSource, /Object\.assign\(previewInputs, nativeRoadRecords\(/);
assert.match(areaBuildSource, /vehicleRoute: optionalFileRecord\(area\.outputs\.vehicleRoute\)/);
const gltf = {
@@ -48,7 +48,7 @@ assert.equal(evaluateGlbBudget(digest, budget).violations[0].key, "glbBytes");
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), "asset-budget-"));
const input = path.join(tempDir, "input.osm");
fs.writeFileSync(input, "<osm/>");
const base = { id: "test-area", input, outputRoot: tempDir };
const base = { id: "test-area", input, outputRoot: tempDir, blender: { roadProvider: "osm2streets" } };
assert.equal(
normalizeAreaConfig(base).outputs.trafficSignals,
path.join(tempDir, "test-area", "osm2streets_web_out", "traffic_signals.json"),
@@ -63,7 +63,7 @@ assert.equal(
);
assert.equal(normalizeAreaConfig({ ...base, budget: { nodes: 800 } }).budget.glbNodes, 800);
assert.equal(normalizeAreaConfig(base).stages.intermediates, false);
assert.equal(normalizeAreaConfig(base).blender.roadProvider, "native");
assert.equal(normalizeAreaConfig(base).blender.roadProvider, "osm2streets");
assert.deepEqual(normalizeAreaConfig(base).v2xPreview, {
enabled: false,
apiBaseUrl: "/api",
@@ -80,7 +80,7 @@ assert.throws(
() => normalizeAreaConfig({ ...base, v2xPreview: "enabled" }),
/v2xPreview must be an object/,
);
assert.equal(normalizeAreaConfig({ ...base, blender: { roadProvider: "native" } }).blender.roadProvider, "native");
assert.equal(normalizeAreaConfig({ ...base, nativeRoadPackage: path.join(tempDir, "roads.zip"), blender: { roadProvider: "native" } }).blender.roadProvider, "native");
assert.throws(
() => normalizeAreaConfig({ ...base, blender: { roadProvider: "other" } }),
/blender\.roadProvider/,
+1 -1
View File
@@ -25,7 +25,7 @@ fs.writeFileSync(path.join(layersDir, "connectors.geojson"), JSON.stringify({ ty
fs.writeFileSync(path.join(layersDir, "vehicle_stop_lines.geojson"), JSON.stringify({ type: "FeatureCollection", features: [{ type: "Feature", properties: { road_id: "road:a" }, geometry: { type: "Polygon", coordinates: [] } }] }));
fs.writeFileSync(path.join(layersDir, "lane_centerlines.geojson"), JSON.stringify({ type: "FeatureCollection", features: roads.map((road) => ({ type: "Feature", properties: { road_id: road.id, lane_index: 1 }, geometry: { type: "LineString", coordinates: road.centerline } })) }));
const descriptor = buildNativeTrafficSimulation({ id: "fixture", outputs: { areaDir, nativeRoadDir } });
const descriptor = buildNativeTrafficSimulation({ id: "fixture", outputs: { areaDir } }, { nativePackage: { rootDir: nativeRoadDir } });
assert.equal(descriptor.schema, SCHEMA);
assert.equal(descriptor.coordinateSystem.route, "WGS84");
assert.ok(descriptor.routes.length >= 1);
+50
View File
@@ -0,0 +1,50 @@
#!/usr/bin/env node
"use strict";
const assert = require("assert");
const fs = require("fs");
const os = require("os");
const path = require("path");
const { zipSync } = require("fflate");
const { importNativeRoadPackage } = require("./lib/native-road-package");
const temp = fs.mkdtempSync(path.join(os.tmpdir(), "native-road-package-"));
const area = { id: "fixture", nativeRoadPackage: "", outputs: { pipelineDir: path.join(temp, "pipeline") } };
const layers = ["road_surface", "connectors"];
const manifest = { contract: "native-road-package/v1.1", areaId: "fixture", layers: [
{ source: "road_surface", role: "surface", materialLayer: "road_surface" },
{ source: "connectors", role: "semantic" },
] };
function validEntries() {
return {
"manifest.json": Buffer.from(JSON.stringify(manifest)),
"compiled.json": Buffer.from("{}"), "diagnostics.json": Buffer.from("{}"), "comparison.json": Buffer.from("{}"),
"traffic-signal-assemblies.json": Buffer.from("{}"), "traffic-signals.json": Buffer.from("{}"),
...Object.fromEntries(layers.map((layer) => [`layers/${layer}.geojson`, Buffer.from('{"type":"FeatureCollection","features":[]}')])),
};
}
function write(name, entries = validEntries()) { const file = path.join(temp, name); fs.writeFileSync(file, zipSync(entries)); area.nativeRoadPackage = file; return file; }
write("valid.zip");
const first = importNativeRoadPackage(area);
assert.equal(first.manifest.areaId, "fixture");
assert.equal(Object.keys(first.records).length, 8);
assert.ok(first.rootDir.includes(first.zipRecord.sha256));
assert.equal(importNativeRoadPackage(area).rootDir, first.rootDir);
assert.throws(() => { area.nativeRoadPackage = path.join(temp, "missing.zip"); importNativeRoadPackage(area); }, /ZIP not found/);
write("corrupt.zip", { "manifest.json": Buffer.from("{") });
assert.throws(() => importNativeRoadPackage(area), /ZIP central directory|Invalid ZIP|Invalid native road manifest|Missing native road ZIP entry/);
write("nested.zip", { "nested/manifest.json": Buffer.from("{}") });
assert.throws(() => importNativeRoadPackage(area), /root-flat/);
write("traversal.zip", { "../manifest.json": Buffer.from("{}") });
assert.throws(() => importNativeRoadPackage(area), /Unsafe ZIP entry/);
const wrongContract = validEntries(); wrongContract["manifest.json"] = Buffer.from(JSON.stringify({ ...manifest, contract: "v1" })); write("contract.zip", wrongContract);
assert.throws(() => importNativeRoadPackage(area), /Unsupported native road contract/);
const wrongArea = validEntries(); wrongArea["manifest.json"] = Buffer.from(JSON.stringify({ ...manifest, areaId: "other" })); write("area.zip", wrongArea);
assert.throws(() => importNativeRoadPackage(area), /areaId mismatch/);
const mismatch = validEntries(); delete mismatch["layers/connectors.geojson"]; write("mismatch.zip", mismatch);
assert.throws(() => importNativeRoadPackage(area), /manifest\/source mismatch/);
const symlink = write("symlink.zip");
const data = fs.readFileSync(symlink); const index = data.indexOf(Buffer.from([0x50, 0x4b, 0x01, 0x02])); data.writeUInt32LE((0xa000 << 16) >>> 0, index + 38); fs.writeFileSync(symlink, data);
assert.throws(() => importNativeRoadPackage(area), /Symbolic links/);
fs.rmSync(temp, { recursive: true, force: true });
console.log("Native road ZIP package tests passed.");
+3 -3
View File
@@ -281,7 +281,7 @@ const previewRuntime = fs.readFileSync(path.join(__dirname, "lib", "cesium-previ
const v2xRuntime = fs.readFileSync(path.join(__dirname, "lib", "v2x-cesium-overlay.js"), "utf8");
const buildAreaSource = fs.readFileSync(path.join(__dirname, "build-area.js"), "utf8");
assert.match(buildAreaSource, /const vehicleModelNames = writeVehicleModel\(area\);/);
assert.doesNotMatch(buildAreaSource, /buildNativeTrafficSimulation/);
assert.match(buildAreaSource, /buildNativeTrafficSimulation/);
const countdownFont = path.join(__dirname, "..", "assets", "fonts", "7LED-1.ttf");
assert.ok(fs.existsSync(countdownFont), "7LED countdown font must be versioned with the project");
assert.doesNotMatch(previewRuntime, /cylinder: \{ length: 6\.7/);
@@ -297,7 +297,7 @@ assert.match(v2xRuntime, /\/network\/ws\/network\/targetPosition/);
assert.match(v2xRuntime, /headingPitchRollQuaternion/);
assert.match(v2xRuntime, /model:/);
assert.match(previewRuntime, /createLiveTrafficSignals\(/);
assert.doesNotMatch(previewRuntime, /const trafficSignals = addTrafficSignals\(/);
assert.match(previewRuntime, /addTrafficSignals\(viewer, signalData, trafficStart, assets\)/);
// The lamp dictionary lives only in the overlay, which hands the preview
// already-normalized {nodeKeys, color, countDown} entries. Two copies of the
@@ -342,7 +342,7 @@ assert.match(v2xRuntime, /Finding the configured V2X intersection/);
assert.match(v2xRuntime, /crossDeviceConfig\/queryList/);
assert.match(v2xRuntime, /gcj02ToWgs84/);
assert.match(previewRuntime, /createLiveVehicleState\(\)/);
assert.doesNotMatch(previewRuntime, /const cruise = addVehicleCruises\(/);
assert.match(previewRuntime, /addVehicleCruises\(viewer, routeData, signalData, trafficStart/);
assert.match(previewRuntime, /new Cesium\.ScreenSpaceEventHandler/);
assert.match(previewRuntime, /vehicleId: record\.id/);
assert.match(previewRuntime, /status === "breakdown"\s+\? "vehicle-breakdown\.png"/);