Refactor road assembly module

2026-08-03 13:39:05 +08:00
parent 64517993e9
commit 0f3c1ee635
12 changed files with 315 additions and 39 deletions
--- a/.trellis/spec/blender/index.md
+++ b/.trellis/spec/blender/index.md
@@ -38,7 +38,7 @@
 │   osmassets/materials.py  材质构建（消费 catalog 的声明）  │
 │   osmassets/tree.py       树实例化                        │
 │   osmassets/water.py grass.py scrub.py                    │
-│   osmassets/building.py fountain.py   要素装配            │
+│   osmassets/building.py fountain.py roads.py   要素装配   │
 │                                                          │
 │   generate_scene.py   export_cesium.py   两个入口         │
 │   tools/scene_digest.py                  结构摘要工具      │
@@ -121,7 +121,7 @@

 | 文件 | 行数 | 层 |
 |---|---|---|
-| `generate_scene.py` | 899 | bpy · 入口 |
+| `generate_scene.py` | 869 | bpy · 入口 |
 | `export_cesium.py` | 647 | bpy · 入口 |
 | `osmassets/tree.py` | 318 | bpy |
 | `osmassets/geom.py` | 241 | 纯 |
@@ -129,7 +129,7 @@
 | `osmassets/catalog.py` | 197 | 纯 |
 | `osmassets/mesh.py` | 126 | bpy |
 | `osmassets/osm.py` | 89 | 纯 |
-| `osmassets/building.py` / `fountain.py` | 56 / 49 | bpy |
+| `osmassets/building.py` / `fountain.py` / `roads.py` | 56 / 49 / 40 | bpy |
 | `osmassets/water.py` / `grass.py` / `scrub.py` | 15 / 22 / 13 | bpy |
 | `tools/scene_digest.py` | 171 | bpy · 工具 |
 | `tests/test_pure.py` | 372 | 纯 · 测试 |
--- a/.trellis/spec/blender/module-structure.md
+++ b/.trellis/spec/blender/module-structure.md
@@ -25,7 +25,7 @@
     osmassets/materials.py  材质构建
     osmassets/tree.py       树实例化
     osmassets/water.py  grass.py  scrub.py
-     osmassets/building.py  fountain.py       要素装配
+     osmassets/building.py  fountain.py  roads.py       要素装配
     generate_scene.py  export_cesium.py      两个入口脚本
     tools/scene_digest.py                    结构摘要工具
 ```
@@ -54,6 +54,7 @@
 | `water.py` / `grass.py` / `scrub.py` | bpy | 单一 OSM 面要素的装配 |
 | `building.py` | bpy | 单一 OSM 面要素 `building=*` 的装配 |
 | `fountain.py` | bpy | 单一 OSM 点要素 `amenity=fountain` 的装配 |
+| `roads.py` | bpy | osm2streets GeoJSON road layer 装配与 OSM highway 折线回退 |

 ### `geom.py` 的两条隐含约定

@@ -111,6 +112,18 @@ def assemble(ring, way_id, tag, office_overrides, collection, materials):
 `counts["industrial_count"]`，以及写 `scene["office_override_way_ids"]`。
 模块只负责 prism / roof / windows / bevel / building custom properties。

+`roads.py` 是 road layer 装配模块，不拥有图层表或计数：
+
+```python
+def assemble_geojson_layer(path, layer_id, projector, collection, material, z): ...
+def assemble_osm_fallback(ways, projector, collection, material): ...
+```
+
+`generate_scene.py` 仍负责创建 `road_mats`、调用 `catalog.check_layers()` 并打印
+`Layer catalog warning:`、遍历 `catalog.ROAD_LAYERS`、维护 `road_counts`，以及写
+`scene["road_feature_counts"]` / `SCENE_DONE["road_features"]`。模块只负责把已选定的
+GeoJSON layer 或 OSM fallback ways 变成 `Road_<layer_id>` / `OSM_Road_<way_id>` 对象。
+
 ### 加一种新 OSM 要素

 目标形态：**新增一个模块 + 注册一行，不改 `build()`**。
--- a/.trellis/spec/guides/artifact-parity-guide.md
+++ b/.trellis/spec/guides/artifact-parity-guide.md
@@ -163,7 +163,7 @@ capturedAt / durationMs / label
 |---|---|---|
 | P0 | 抽纯函数到 `osmassets/{osm,geom}.py` | ✅ 已完成 |
 | P1 | `catalog.py` 单一定义源 + `check_layers` | ✅ 已完成 |
-| P2 | 要素注册表 | ⚠️ **部分**——`water/grass/scrub/tree/fountain/building.py` 已拆出，但**没有 `features/` 注册表**，`roads` 仍在 `generate_scene.py` 里 |
+| P2 | 要素注册表 | ⚠️ **部分**——`water/grass/scrub/tree/fountain/building/roads.py` 已拆出，但**没有 `features/` 注册表** |
 | P3 | 材质契约化（自定义属性传递 spec） | ✅ 已完成——`catalog.MATERIALS[*]["cesium"]` 经 `materials.from_spec()` 写入 `material["cesium_export"]`，`export_cesium.py` 优先读该属性；四张材质名表仅作旧 `.blend` 回退 |

 ### 已知缺陷（记录在案，本轮不修）
--- a/.trellis/tasks/08-03-extract-road-module/check.jsonl
+++ b/.trellis/tasks/08-03-extract-road-module/check.jsonl
@@ -0,0 +1,4 @@
+{"file": ".trellis/spec/blender/testing.md", "reason": "Defines pure test command and bpy-layer parity expectations for validation."}
+{"file": ".trellis/spec/guides/artifact-parity-guide.md", "reason": "Defines compare contract and expected no-diff result for refactors."}
+{"file": ".trellis/spec/pipeline/layer-registry.md", "reason": "Defines ROAD_LAYERS order and warning behavior to review after extraction."}
+{"file": ".trellis/spec/blender/module-structure.md", "reason": "Defines dependency direction and safe import boundaries to review after new module creation."}
--- a/.trellis/tasks/08-03-extract-road-module/design.md
+++ b/.trellis/tasks/08-03-extract-road-module/design.md
@@ -0,0 +1,74 @@
+# Design
+
+## Architecture
+
+`roads.py` will join the bpy-layer feature modules under
+`blender/osmassets/`. It may import `os`, `json`, `MeshBatch`, `add_polyline`,
+and geometry helpers required by current road assembly:
+
+```python
+from osmassets.geom import clip_polygon, feature_in_bounds, geometry_rings
+from osmassets.mesh import MeshBatch, add_polyline
+```
+
+The module will expose two functions:
+
+```python
+def assemble_geojson_layer(path, layer_id, projector, collection, material, z):
+    ...
+
+def assemble_osm_fallback(ways, projector, collection, material):
+    ...
+```
+
+`assemble_geojson_layer()` returns the same count as the current
+`add_geojson_layer()`. `assemble_osm_fallback()` returns no count because the
+current fallback also leaves `road_counts` unchanged.
+
+## Boundaries
+
+- `generate_scene.py` keeps catalog ownership: `road_mats`, `ROAD_LAYERS`
+  iteration, `check_layers()` warnings, `road_counts`, scene metadata, and
+  stdout markers.
+- `roads.py` owns only object/mesh construction for already chosen road inputs.
+- `catalog.py` and `scripts/lib/scene-layers.js` remain untouched; road layer
+  collection/order is not part of this task.
+- No new pure-Python logic is introduced. Existing geometry helpers are reused.
+
+## Data Flow
+
+1. `generate_scene.py` creates `road_mats` from `catalog.ROAD_LAYERS` and
+   `catalog.road_material_specs()`.
+2. If `geojson_dir` exists, `generate_scene.py` runs `catalog.check_layers()`
+   and prints warnings exactly as before.
+3. For each `catalog.ROAD_LAYERS` entry, `generate_scene.py` calls
+   `roads.assemble_geojson_layer(...)` and stores the returned count under the
+   layer id in `road_counts`.
+4. If `road_counts.get("road_surface", 0) == 0`, `generate_scene.py` calls
+   `roads.assemble_osm_fallback(...)` with OSM ways and the road surface
+   material.
+5. `generate_scene.py` writes `road_counts` into scene metadata and `SCENE_DONE`.
+
+## Compatibility
+
+The move must preserve road object names, mesh names, geometry clipping,
+feature counting, layer iteration order, warning behavior, fallback trigger,
+fallback widths, z values, material assignments, and metadata JSON. Cesium
+export consumes the resulting `.blend`, so parity should see no non-ignored
+contract difference.
+
+## Trade-Offs
+
+`catalog.check_layers()` intentionally stays in `generate_scene.py`. Moving it
+into `roads.py` would mix cross-language catalog validation with object
+assembly and increase the blast radius of this refactor.
+
+The full feature registry remains deferred. Roads are the last high-risk
+assembly extraction; registry design should be evaluated after this commit when
+the remaining `generate_scene.py` responsibilities are clearer.
+
+## Rollback
+
+Rollback is mechanical: move `assemble_geojson_layer()` and
+`assemble_osm_fallback()` logic back into `generate_scene.py`, restore the
+direct call sites, remove the `roads.py` import, and delete the module.
--- a/.trellis/tasks/08-03-extract-road-module/implement.jsonl
+++ b/.trellis/tasks/08-03-extract-road-module/implement.jsonl
@@ -0,0 +1,5 @@
+{"file": ".trellis/spec/blender/module-structure.md", "reason": "Defines bpy-layer module boundaries and feature module responsibilities for extracting roads."}
+{"file": ".trellis/spec/blender/asset-generation.md", "reason": "Defines MeshBatch usage, material order sensitivity, and deterministic asset-generation constraints."}
+{"file": ".trellis/spec/pipeline/layer-registry.md", "reason": "Defines ROAD_LAYERS / SCENE_LAYERS cross-language ordering contract and check_layers warning behavior."}
+{"file": ".trellis/spec/guides/artifact-parity-guide.md", "reason": "Defines mandatory before/after parity validation for pure Blender scene refactors."}
+{"file": ".trellis/spec/guides/cross-layer-thinking-guide.md", "reason": "Defines cross-layer road layer and stage-output contract risks."}
--- a/.trellis/tasks/08-03-extract-road-module/implement.md
+++ b/.trellis/tasks/08-03-extract-road-module/implement.md
@@ -0,0 +1,46 @@
+# Implementation Plan
+
+## Checklist
+
+1. Capture a parity baseline before product-code edits.
+2. Add `blender/osmassets/roads.py` with current GeoJSON road layer assembly
+   and OSM fallback logic.
+3. Import `roads.py` in `blender/generate_scene.py`.
+4. Remove inline `add_geojson_layer()` from `generate_scene.py`.
+5. Replace the GeoJSON road layer call site with
+   `roads.assemble_geojson_layer(...)`.
+6. Replace the OSM fallback loop with `roads.assemble_osm_fallback(...)`.
+7. Run Python validation:
+   - `python3 -m py_compile blender/osmassets/roads.py`
+   - `python3 -m py_compile blender/generate_scene.py`
+   - `python3 -m unittest blender/tests/test_pure.py`
+8. Capture parity after the refactor.
+9. Compare before/after parity snapshots and inspect any non-ignored diff.
+10. Update `.trellis/spec` and `docs/changelog.md` only for durable status or
+    convention changes discovered during implementation.
+11. Commit, archive the task, and record the journal entry.
+
+## Validation Commands
+
+```bash
+node scripts/parity.js capture road-module-before --stages blender,cesium
+python3 -m py_compile blender/osmassets/roads.py
+python3 -m py_compile blender/generate_scene.py
+python3 -m unittest blender/tests/test_pure.py
+node scripts/parity.js capture road-module-after --stages blender,cesium
+node scripts/parity.js compare road-module-before road-module-after
+```
+
+## Risk Points
+
+- `ROAD_LAYERS` order is load-bearing. Do not reorder, filter, or copy it into
+  a second list.
+- `road_counts` must remain owned by `generate_scene.py`; fallback roads must
+  still leave it unchanged.
+- `Layer catalog warning:` print text must remain unchanged because it is the
+  only runtime warning for JS/Python layer drift.
+- `Road_<layer_id>` and `OSM_Road_<way_id>` names affect parity digests.
+- `clip_polygon()` and `feature_in_bounds()` behavior must be reused exactly;
+  do not replace with new geometry logic.
+- Parity is mandatory because this is a pure refactor of bpy-layer scene
+  generation.
--- a/.trellis/tasks/08-03-extract-road-module/prd.md
+++ b/.trellis/tasks/08-03-extract-road-module/prd.md
@@ -0,0 +1,93 @@
+# Extract road module
+
+## Goal
+
+Move Blender road assembly out of `blender/generate_scene.py` into a dedicated
+bpy-layer module, keeping generated `.blend` / `.glb` / metadata structure
+unchanged. This completes the P2 feature-module extraction except for the
+deferred full `features/` registry.
+
+## Background
+
+- `.trellis/spec/guides/artifact-parity-guide.md` records P2 as partial:
+  `water.py`, `grass.py`, `scrub.py`, `tree.py`, `fountain.py`, and
+  `building.py` are already split out, while roads still live in
+  `generate_scene.py`.
+- `blender/generate_scene.py:148` defines `add_geojson_layer()`, which reads
+  one osm2streets GeoJSON road layer, filters by bounds, projects coordinates,
+  clips polygons, batches them as `Road_<layer>`, and returns a feature count.
+- `blender/generate_scene.py:656` creates `road_mats` by zipping
+  `catalog.ROAD_LAYERS` with `catalog.road_material_specs()`.
+- `blender/generate_scene.py:741` runs `catalog.check_layers(geojson_dir)` and
+  prints `Layer catalog warning:` messages before iterating `ROAD_LAYERS`.
+- `blender/generate_scene.py:750` falls back to simple OSM highway polylines
+  when `road_counts["road_surface"]` is zero.
+- `blender/generate_scene.py:866` stores `scene["road_feature_counts"]`; the
+  same `road_counts` object is emitted in `SCENE_DONE` as `road_features`.
+
+## Requirements
+
+1. Add a new bpy-layer module at `blender/osmassets/roads.py`.
+2. Move the current roads assembly behavior into the new module:
+   - GeoJSON layer mesh assembly currently in `add_geojson_layer()`;
+   - simple OSM highway fallback currently in the `road_surface == 0` block.
+3. Keep `generate_scene.py` responsible for:
+   - creating `road_mats` from `catalog.ROAD_LAYERS` and
+     `catalog.road_material_specs()`;
+   - running `catalog.check_layers()` and printing existing warning text;
+   - owning `road_counts`;
+   - writing `scene["road_feature_counts"]`;
+   - writing the existing `SCENE_DONE` JSON shape.
+4. Preserve existing road object names and mesh names:
+   - `Road_<layer_id>`
+   - `OSM_Road_<way_id>`
+5. Preserve existing GeoJSON behavior:
+   - missing layer file returns count `0`;
+   - feature is skipped when `feature_in_bounds()` is false;
+   - rings come from `geometry_rings()`;
+   - projected rings are clipped with `clip_polygon()`;
+   - only rings with at least three points are batched and counted.
+6. Preserve existing OSM fallback behavior:
+   - fallback triggers only when `road_counts.get("road_surface", 0) == 0`;
+   - only ways with a `highway` tag and at least two coordinates are drawn;
+   - widths remain `secondary: 7.0`, `residential: 5.5`, `service: 3.5`,
+     default `4.0`;
+   - fallback material remains `road_mats["road_surface"]`;
+   - fallback z remains `0.03`;
+   - fallback does not add to `road_counts`.
+7. Do not introduce a full `features/` registry in this task.
+8. Do not modify `ROAD_LAYERS`, `SCENE_LAYERS`, material definitions,
+   `MATERIALS` order, GeoJSON schema, z-index/z-height values, stdout markers,
+   parity ignore lists, or known unrelated defects D1-D3.
+
+## Acceptance Criteria
+
+- [ ] `blender/osmassets/roads.py` contains road assembly behavior and imports
+      only bpy-layer-safe dependencies.
+- [ ] `blender/generate_scene.py` imports and calls the new roads module; no
+      inline `add_geojson_layer()` remains there.
+- [ ] `road_counts`, `scene["road_feature_counts"]`, and `SCENE_DONE`
+      `road_features` remain unchanged.
+- [ ] Existing `Layer catalog warning:` print behavior remains in
+      `generate_scene.py`.
+- [ ] `python3 -m py_compile blender/osmassets/roads.py` passes.
+- [ ] `python3 -m py_compile blender/generate_scene.py` passes.
+- [ ] `python3 -m unittest blender/tests/test_pure.py` passes.
+- [ ] A before/after parity comparison is run for the Blender/Cesium stages and
+      any non-ignored diff is either absent or explicitly explained as
+      expected. For this refactor, the expected result is no contract diff.
+
+## Out Of Scope
+
+- Full `features/` registry implementation.
+- Any change to osm2streets road splitting, `SCENE_LAYERS`, or QGIS pipeline
+  behavior.
+- Any change to `catalog.ROAD_LAYERS`, material names, color values, or order.
+- Any change to fallback highway widths, z values, object names, or counts.
+- Turning `catalog.check_layers()` warnings into errors.
+- Moving `road_mats` creation or `road_feature_counts` scene metadata out of
+  `generate_scene.py`.
+
+## Open Questions
+
+None.
--- a/.trellis/tasks/08-03-extract-road-module/task.json
+++ b/.trellis/tasks/08-03-extract-road-module/task.json
@@ -0,0 +1,26 @@
+{
+  "id": "extract-road-module",
+  "name": "extract-road-module",
+  "title": "Extract road module",
+  "description": "Move Blender road assembly out of generate_scene.py into a dedicated bpy-layer module without changing generated artifacts.",
+  "status": "in_progress",
+  "dev_type": null,
+  "scope": null,
+  "package": null,
+  "priority": "P2",
+  "creator": "dingkang",
+  "assignee": "dingkang",
+  "createdAt": "2026-08-03",
+  "completedAt": null,
+  "branch": null,
+  "base_branch": "main",
+  "worktree_path": null,
+  "commit": null,
+  "pr_url": null,
+  "subtasks": [],
+  "children": [],
+  "parent": null,
+  "relatedFiles": [],
+  "notes": "",
+  "meta": {}
+}
--- a/blender/generate_scene.py
+++ b/blender/generate_scene.py
@@ -35,10 +35,7 @@ if _HERE not in sys.path:

 from osmassets import catalog  # noqa: E402
 from osmassets.geom import (  # noqa: E402  (needs the sys.path line above)
-    clip_polygon,
    distance_to_ring,
-    feature_in_bounds,
-    geometry_rings,
    point_in_polygon,
    polygon_area,
    sample_polygon_interior,
@@ -51,7 +48,6 @@ from osmassets.materials import (  # noqa: E402
 )
 from osmassets.mesh import (  # noqa: E402
    MeshBatch,
-    add_polyline,
    new_collection,
 )
 from osmassets.osm import Projector, parse_height, parse_osm  # noqa: E402
@@ -59,6 +55,7 @@ from osmassets import building as _building  # noqa: E402
 from osmassets import fountain as _fountain  # noqa: E402
 from osmassets import water as _water  # noqa: E402
 from osmassets import grass as _grass  # noqa: E402
+from osmassets import roads as _roads  # noqa: E402
 from osmassets import scrub as _scrub  # noqa: E402
 from osmassets import tree as _tree  # noqa: E402

@@ -145,29 +142,6 @@ def cli_args():
    return values


-def add_geojson_layer(path, layer, projector, collection, material, z):
-    if not os.path.exists(path):
-        return 0
-    with open(path, "r", encoding="utf-8") as handle:
-        data = json.load(handle)
-    batch = MeshBatch("Road_" + layer, collection, material)
-    b = projector.bounds
-    xmin, ymin = projector.xy((b["min_lon"], b["min_lat"]))
-    xmax, ymax = projector.xy((b["max_lon"], b["max_lat"]))
-    count = 0
-    for feature in data.get("features", []):
-        if not feature_in_bounds(feature, projector):
-            continue
-        for ring in geometry_rings(feature.get("geometry")):
-            points = [projector.xy(pair) for pair in ring]
-            points = clip_polygon(points, xmin, xmax, ymin, ymax)
-            if len(points) >= 3:
-                batch.add_polygon(points, z)
-                count += 1
-    batch.finish()
-    return count
-
-
 def add_tree_batch(positions, collection, trunk_material, leaf_material):
    trunk = MeshBatch("Tree_Trunks", collection, trunk_material)
    leaves = MeshBatch("Tree_Crowns", collection, leaf_material)
@@ -743,17 +717,13 @@ def build(args):
            print("Layer catalog warning:", problem)
        for layer in catalog.ROAD_LAYERS:
            layer_id = layer["id"]
-            road_counts[layer_id] = add_geojson_layer(
+            road_counts[layer_id] = _roads.assemble_geojson_layer(
                os.path.join(geojson_dir, layer_id + ".geojson"), layer_id,
                projector, roads_c, road_mats[layer_id], layer["z"])

    if road_counts.get("road_surface", 0) == 0:
-        for way in ways:
-            highway = way["tags"].get("highway")
-            if highway and len(way["coords"]) >= 2:
-                width = {"secondary": 7.0, "residential": 5.5, "service": 3.5}.get(highway, 4.0)
-                add_polyline("OSM_Road_" + str(way["id"]), way["coords"], projector,
-                             roads_c, road_mats["road_surface"], width, 0.03)
+        _roads.assemble_osm_fallback(
+            ways, projector, roads_c, road_mats["road_surface"])

    trees = []
    individual_tree_count = 0
--- a/blender/osmassets/roads.py
+++ b/blender/osmassets/roads.py
@@ -0,0 +1,40 @@
+"""Road layer assembly from osm2streets GeoJSON and OSM fallback ways."""
+
+import json
+import os
+
+from osmassets.geom import clip_polygon, feature_in_bounds, geometry_rings
+from osmassets.mesh import MeshBatch, add_polyline
+
+
+def assemble_geojson_layer(path, layer_id, projector, collection, material, z):
+    if not os.path.exists(path):
+        return 0
+    with open(path, "r", encoding="utf-8") as handle:
+        data = json.load(handle)
+    batch = MeshBatch("Road_" + layer_id, collection, material)
+    b = projector.bounds
+    xmin, ymin = projector.xy((b["min_lon"], b["min_lat"]))
+    xmax, ymax = projector.xy((b["max_lon"], b["max_lat"]))
+    count = 0
+    for feature in data.get("features", []):
+        if not feature_in_bounds(feature, projector):
+            continue
+        for ring in geometry_rings(feature.get("geometry")):
+            points = [projector.xy(pair) for pair in ring]
+            points = clip_polygon(points, xmin, xmax, ymin, ymax)
+            if len(points) >= 3:
+                batch.add_polygon(points, z)
+                count += 1
+    batch.finish()
+    return count
+
+
+def assemble_osm_fallback(ways, projector, collection, material):
+    for way in ways:
+        highway = way["tags"].get("highway")
+        if highway and len(way["coords"]) >= 2:
+            width = {"secondary": 7.0, "residential": 5.5,
+                     "service": 3.5}.get(highway, 4.0)
+            add_polyline("OSM_Road_" + str(way["id"]), way["coords"], projector,
+                         collection, material, width, 0.03)
--- a/docs/changelog.md
+++ b/docs/changelog.md
@@ -22,6 +22,11 @@
  roof、window bands、bevel 和 building custom properties；`generate_scene.py` 继续
  负责 `office_overrides`、材质创建、way 分发、计数和 scene metadata。Blender/Cesium
  parity before/after 在两个样本上均 identical；P2 剩余未拆出部分更新为 `roads`。
+- 抽出 roads 要素装配：新增 `blender/osmassets/roads.py`，迁移 osm2streets GeoJSON
+  road layer mesh assembly 与 OSM highway fallback polyline 装配；`generate_scene.py`
+  继续负责 `ROAD_LAYERS` 遍历、`catalog.check_layers()` warning、`road_counts`、scene
+  metadata 和 `SCENE_DONE` JSON。Blender/Cesium parity before/after 在两个样本上均
+  identical；P2 剩余工作更新为完整 `features/` 注册表设计。

 ## 2026-07-31（三）远看发黑的真正原因：反照率没被提亮