Refactor road assembly module

This commit is contained in:
2026-08-03 13:39:05 +08:00
parent 64517993e9
commit 0f3c1ee635
12 changed files with 315 additions and 39 deletions

View File

@@ -38,7 +38,7 @@
│ osmassets/materials.py 材质构建(消费 catalog 的声明) │ │ osmassets/materials.py 材质构建(消费 catalog 的声明) │
│ osmassets/tree.py 树实例化 │ │ osmassets/tree.py 树实例化 │
│ osmassets/water.py grass.py scrub.py │ │ osmassets/water.py grass.py scrub.py │
│ osmassets/building.py fountain.py 要素装配 │ osmassets/building.py fountain.py roads.py 要素装配 │
│ │ │ │
│ generate_scene.py export_cesium.py 两个入口 │ │ generate_scene.py export_cesium.py 两个入口 │
│ tools/scene_digest.py 结构摘要工具 │ │ tools/scene_digest.py 结构摘要工具 │
@@ -121,7 +121,7 @@
| 文件 | 行数 | 层 | | 文件 | 行数 | 层 |
|---|---|---| |---|---|---|
| `generate_scene.py` | 899 | bpy · 入口 | | `generate_scene.py` | 869 | bpy · 入口 |
| `export_cesium.py` | 647 | bpy · 入口 | | `export_cesium.py` | 647 | bpy · 入口 |
| `osmassets/tree.py` | 318 | bpy | | `osmassets/tree.py` | 318 | bpy |
| `osmassets/geom.py` | 241 | 纯 | | `osmassets/geom.py` | 241 | 纯 |
@@ -129,7 +129,7 @@
| `osmassets/catalog.py` | 197 | 纯 | | `osmassets/catalog.py` | 197 | 纯 |
| `osmassets/mesh.py` | 126 | bpy | | `osmassets/mesh.py` | 126 | bpy |
| `osmassets/osm.py` | 89 | 纯 | | `osmassets/osm.py` | 89 | 纯 |
| `osmassets/building.py` / `fountain.py` | 56 / 49 | bpy | | `osmassets/building.py` / `fountain.py` / `roads.py` | 56 / 49 / 40 | bpy |
| `osmassets/water.py` / `grass.py` / `scrub.py` | 15 / 22 / 13 | bpy | | `osmassets/water.py` / `grass.py` / `scrub.py` | 15 / 22 / 13 | bpy |
| `tools/scene_digest.py` | 171 | bpy · 工具 | | `tools/scene_digest.py` | 171 | bpy · 工具 |
| `tests/test_pure.py` | 372 | 纯 · 测试 | | `tests/test_pure.py` | 372 | 纯 · 测试 |

View File

@@ -25,7 +25,7 @@
osmassets/materials.py 材质构建 osmassets/materials.py 材质构建
osmassets/tree.py 树实例化 osmassets/tree.py 树实例化
osmassets/water.py grass.py scrub.py osmassets/water.py grass.py scrub.py
osmassets/building.py fountain.py 要素装配 osmassets/building.py fountain.py roads.py 要素装配
generate_scene.py export_cesium.py 两个入口脚本 generate_scene.py export_cesium.py 两个入口脚本
tools/scene_digest.py 结构摘要工具 tools/scene_digest.py 结构摘要工具
``` ```
@@ -54,6 +54,7 @@
| `water.py` / `grass.py` / `scrub.py` | bpy | 单一 OSM 面要素的装配 | | `water.py` / `grass.py` / `scrub.py` | bpy | 单一 OSM 面要素的装配 |
| `building.py` | bpy | 单一 OSM 面要素 `building=*` 的装配 | | `building.py` | bpy | 单一 OSM 面要素 `building=*` 的装配 |
| `fountain.py` | bpy | 单一 OSM 点要素 `amenity=fountain` 的装配 | | `fountain.py` | bpy | 单一 OSM 点要素 `amenity=fountain` 的装配 |
| `roads.py` | bpy | osm2streets GeoJSON road layer 装配与 OSM highway 折线回退 |
### `geom.py` 的两条隐含约定 ### `geom.py` 的两条隐含约定
@@ -111,6 +112,18 @@ def assemble(ring, way_id, tag, office_overrides, collection, materials):
`counts["industrial_count"]`,以及写 `scene["office_override_way_ids"]` `counts["industrial_count"]`,以及写 `scene["office_override_way_ids"]`
模块只负责 prism / roof / windows / bevel / building custom properties。 模块只负责 prism / roof / windows / bevel / building custom properties。
`roads.py` 是 road layer 装配模块,不拥有图层表或计数:
```python
def assemble_geojson_layer(path, layer_id, projector, collection, material, z): ...
def assemble_osm_fallback(ways, projector, collection, material): ...
```
`generate_scene.py` 仍负责创建 `road_mats`、调用 `catalog.check_layers()` 并打印
`Layer catalog warning:`、遍历 `catalog.ROAD_LAYERS`、维护 `road_counts`,以及写
`scene["road_feature_counts"]` / `SCENE_DONE["road_features"]`。模块只负责把已选定的
GeoJSON layer 或 OSM fallback ways 变成 `Road_<layer_id>` / `OSM_Road_<way_id>` 对象。
### 加一种新 OSM 要素 ### 加一种新 OSM 要素
目标形态:**新增一个模块 + 注册一行,不改 `build()`**。 目标形态:**新增一个模块 + 注册一行,不改 `build()`**。

View File

@@ -163,7 +163,7 @@ capturedAt / durationMs / label
|---|---|---| |---|---|---|
| P0 | 抽纯函数到 `osmassets/{osm,geom}.py` | ✅ 已完成 | | P0 | 抽纯函数到 `osmassets/{osm,geom}.py` | ✅ 已完成 |
| P1 | `catalog.py` 单一定义源 + `check_layers` | ✅ 已完成 | | P1 | `catalog.py` 单一定义源 + `check_layers` | ✅ 已完成 |
| P2 | 要素注册表 | ⚠️ **部分**——`water/grass/scrub/tree/fountain/building.py` 已拆出,但**没有 `features/` 注册表**`roads` 仍在 `generate_scene.py` | | P2 | 要素注册表 | ⚠️ **部分**——`water/grass/scrub/tree/fountain/building/roads.py` 已拆出,但**没有 `features/` 注册表** |
| P3 | 材质契约化(自定义属性传递 spec | ✅ 已完成——`catalog.MATERIALS[*]["cesium"]``materials.from_spec()` 写入 `material["cesium_export"]``export_cesium.py` 优先读该属性;四张材质名表仅作旧 `.blend` 回退 | | P3 | 材质契约化(自定义属性传递 spec | ✅ 已完成——`catalog.MATERIALS[*]["cesium"]``materials.from_spec()` 写入 `material["cesium_export"]``export_cesium.py` 优先读该属性;四张材质名表仅作旧 `.blend` 回退 |
### 已知缺陷(记录在案,本轮不修) ### 已知缺陷(记录在案,本轮不修)

View File

@@ -0,0 +1,4 @@
{"file": ".trellis/spec/blender/testing.md", "reason": "Defines pure test command and bpy-layer parity expectations for validation."}
{"file": ".trellis/spec/guides/artifact-parity-guide.md", "reason": "Defines compare contract and expected no-diff result for refactors."}
{"file": ".trellis/spec/pipeline/layer-registry.md", "reason": "Defines ROAD_LAYERS order and warning behavior to review after extraction."}
{"file": ".trellis/spec/blender/module-structure.md", "reason": "Defines dependency direction and safe import boundaries to review after new module creation."}

View File

@@ -0,0 +1,74 @@
# Design
## Architecture
`roads.py` will join the bpy-layer feature modules under
`blender/osmassets/`. It may import `os`, `json`, `MeshBatch`, `add_polyline`,
and geometry helpers required by current road assembly:
```python
from osmassets.geom import clip_polygon, feature_in_bounds, geometry_rings
from osmassets.mesh import MeshBatch, add_polyline
```
The module will expose two functions:
```python
def assemble_geojson_layer(path, layer_id, projector, collection, material, z):
...
def assemble_osm_fallback(ways, projector, collection, material):
...
```
`assemble_geojson_layer()` returns the same count as the current
`add_geojson_layer()`. `assemble_osm_fallback()` returns no count because the
current fallback also leaves `road_counts` unchanged.
## Boundaries
- `generate_scene.py` keeps catalog ownership: `road_mats`, `ROAD_LAYERS`
iteration, `check_layers()` warnings, `road_counts`, scene metadata, and
stdout markers.
- `roads.py` owns only object/mesh construction for already chosen road inputs.
- `catalog.py` and `scripts/lib/scene-layers.js` remain untouched; road layer
collection/order is not part of this task.
- No new pure-Python logic is introduced. Existing geometry helpers are reused.
## Data Flow
1. `generate_scene.py` creates `road_mats` from `catalog.ROAD_LAYERS` and
`catalog.road_material_specs()`.
2. If `geojson_dir` exists, `generate_scene.py` runs `catalog.check_layers()`
and prints warnings exactly as before.
3. For each `catalog.ROAD_LAYERS` entry, `generate_scene.py` calls
`roads.assemble_geojson_layer(...)` and stores the returned count under the
layer id in `road_counts`.
4. If `road_counts.get("road_surface", 0) == 0`, `generate_scene.py` calls
`roads.assemble_osm_fallback(...)` with OSM ways and the road surface
material.
5. `generate_scene.py` writes `road_counts` into scene metadata and `SCENE_DONE`.
## Compatibility
The move must preserve road object names, mesh names, geometry clipping,
feature counting, layer iteration order, warning behavior, fallback trigger,
fallback widths, z values, material assignments, and metadata JSON. Cesium
export consumes the resulting `.blend`, so parity should see no non-ignored
contract difference.
## Trade-Offs
`catalog.check_layers()` intentionally stays in `generate_scene.py`. Moving it
into `roads.py` would mix cross-language catalog validation with object
assembly and increase the blast radius of this refactor.
The full feature registry remains deferred. Roads are the last high-risk
assembly extraction; registry design should be evaluated after this commit when
the remaining `generate_scene.py` responsibilities are clearer.
## Rollback
Rollback is mechanical: move `assemble_geojson_layer()` and
`assemble_osm_fallback()` logic back into `generate_scene.py`, restore the
direct call sites, remove the `roads.py` import, and delete the module.

View File

@@ -0,0 +1,5 @@
{"file": ".trellis/spec/blender/module-structure.md", "reason": "Defines bpy-layer module boundaries and feature module responsibilities for extracting roads."}
{"file": ".trellis/spec/blender/asset-generation.md", "reason": "Defines MeshBatch usage, material order sensitivity, and deterministic asset-generation constraints."}
{"file": ".trellis/spec/pipeline/layer-registry.md", "reason": "Defines ROAD_LAYERS / SCENE_LAYERS cross-language ordering contract and check_layers warning behavior."}
{"file": ".trellis/spec/guides/artifact-parity-guide.md", "reason": "Defines mandatory before/after parity validation for pure Blender scene refactors."}
{"file": ".trellis/spec/guides/cross-layer-thinking-guide.md", "reason": "Defines cross-layer road layer and stage-output contract risks."}

View File

@@ -0,0 +1,46 @@
# Implementation Plan
## Checklist
1. Capture a parity baseline before product-code edits.
2. Add `blender/osmassets/roads.py` with current GeoJSON road layer assembly
and OSM fallback logic.
3. Import `roads.py` in `blender/generate_scene.py`.
4. Remove inline `add_geojson_layer()` from `generate_scene.py`.
5. Replace the GeoJSON road layer call site with
`roads.assemble_geojson_layer(...)`.
6. Replace the OSM fallback loop with `roads.assemble_osm_fallback(...)`.
7. Run Python validation:
- `python3 -m py_compile blender/osmassets/roads.py`
- `python3 -m py_compile blender/generate_scene.py`
- `python3 -m unittest blender/tests/test_pure.py`
8. Capture parity after the refactor.
9. Compare before/after parity snapshots and inspect any non-ignored diff.
10. Update `.trellis/spec` and `docs/changelog.md` only for durable status or
convention changes discovered during implementation.
11. Commit, archive the task, and record the journal entry.
## Validation Commands
```bash
node scripts/parity.js capture road-module-before --stages blender,cesium
python3 -m py_compile blender/osmassets/roads.py
python3 -m py_compile blender/generate_scene.py
python3 -m unittest blender/tests/test_pure.py
node scripts/parity.js capture road-module-after --stages blender,cesium
node scripts/parity.js compare road-module-before road-module-after
```
## Risk Points
- `ROAD_LAYERS` order is load-bearing. Do not reorder, filter, or copy it into
a second list.
- `road_counts` must remain owned by `generate_scene.py`; fallback roads must
still leave it unchanged.
- `Layer catalog warning:` print text must remain unchanged because it is the
only runtime warning for JS/Python layer drift.
- `Road_<layer_id>` and `OSM_Road_<way_id>` names affect parity digests.
- `clip_polygon()` and `feature_in_bounds()` behavior must be reused exactly;
do not replace with new geometry logic.
- Parity is mandatory because this is a pure refactor of bpy-layer scene
generation.

View File

@@ -0,0 +1,93 @@
# Extract road module
## Goal
Move Blender road assembly out of `blender/generate_scene.py` into a dedicated
bpy-layer module, keeping generated `.blend` / `.glb` / metadata structure
unchanged. This completes the P2 feature-module extraction except for the
deferred full `features/` registry.
## Background
- `.trellis/spec/guides/artifact-parity-guide.md` records P2 as partial:
`water.py`, `grass.py`, `scrub.py`, `tree.py`, `fountain.py`, and
`building.py` are already split out, while roads still live in
`generate_scene.py`.
- `blender/generate_scene.py:148` defines `add_geojson_layer()`, which reads
one osm2streets GeoJSON road layer, filters by bounds, projects coordinates,
clips polygons, batches them as `Road_<layer>`, and returns a feature count.
- `blender/generate_scene.py:656` creates `road_mats` by zipping
`catalog.ROAD_LAYERS` with `catalog.road_material_specs()`.
- `blender/generate_scene.py:741` runs `catalog.check_layers(geojson_dir)` and
prints `Layer catalog warning:` messages before iterating `ROAD_LAYERS`.
- `blender/generate_scene.py:750` falls back to simple OSM highway polylines
when `road_counts["road_surface"]` is zero.
- `blender/generate_scene.py:866` stores `scene["road_feature_counts"]`; the
same `road_counts` object is emitted in `SCENE_DONE` as `road_features`.
## Requirements
1. Add a new bpy-layer module at `blender/osmassets/roads.py`.
2. Move the current roads assembly behavior into the new module:
- GeoJSON layer mesh assembly currently in `add_geojson_layer()`;
- simple OSM highway fallback currently in the `road_surface == 0` block.
3. Keep `generate_scene.py` responsible for:
- creating `road_mats` from `catalog.ROAD_LAYERS` and
`catalog.road_material_specs()`;
- running `catalog.check_layers()` and printing existing warning text;
- owning `road_counts`;
- writing `scene["road_feature_counts"]`;
- writing the existing `SCENE_DONE` JSON shape.
4. Preserve existing road object names and mesh names:
- `Road_<layer_id>`
- `OSM_Road_<way_id>`
5. Preserve existing GeoJSON behavior:
- missing layer file returns count `0`;
- feature is skipped when `feature_in_bounds()` is false;
- rings come from `geometry_rings()`;
- projected rings are clipped with `clip_polygon()`;
- only rings with at least three points are batched and counted.
6. Preserve existing OSM fallback behavior:
- fallback triggers only when `road_counts.get("road_surface", 0) == 0`;
- only ways with a `highway` tag and at least two coordinates are drawn;
- widths remain `secondary: 7.0`, `residential: 5.5`, `service: 3.5`,
default `4.0`;
- fallback material remains `road_mats["road_surface"]`;
- fallback z remains `0.03`;
- fallback does not add to `road_counts`.
7. Do not introduce a full `features/` registry in this task.
8. Do not modify `ROAD_LAYERS`, `SCENE_LAYERS`, material definitions,
`MATERIALS` order, GeoJSON schema, z-index/z-height values, stdout markers,
parity ignore lists, or known unrelated defects D1-D3.
## Acceptance Criteria
- [ ] `blender/osmassets/roads.py` contains road assembly behavior and imports
only bpy-layer-safe dependencies.
- [ ] `blender/generate_scene.py` imports and calls the new roads module; no
inline `add_geojson_layer()` remains there.
- [ ] `road_counts`, `scene["road_feature_counts"]`, and `SCENE_DONE`
`road_features` remain unchanged.
- [ ] Existing `Layer catalog warning:` print behavior remains in
`generate_scene.py`.
- [ ] `python3 -m py_compile blender/osmassets/roads.py` passes.
- [ ] `python3 -m py_compile blender/generate_scene.py` passes.
- [ ] `python3 -m unittest blender/tests/test_pure.py` passes.
- [ ] A before/after parity comparison is run for the Blender/Cesium stages and
any non-ignored diff is either absent or explicitly explained as
expected. For this refactor, the expected result is no contract diff.
## Out Of Scope
- Full `features/` registry implementation.
- Any change to osm2streets road splitting, `SCENE_LAYERS`, or QGIS pipeline
behavior.
- Any change to `catalog.ROAD_LAYERS`, material names, color values, or order.
- Any change to fallback highway widths, z values, object names, or counts.
- Turning `catalog.check_layers()` warnings into errors.
- Moving `road_mats` creation or `road_feature_counts` scene metadata out of
`generate_scene.py`.
## Open Questions
None.

View File

@@ -0,0 +1,26 @@
{
"id": "extract-road-module",
"name": "extract-road-module",
"title": "Extract road module",
"description": "Move Blender road assembly out of generate_scene.py into a dedicated bpy-layer module without changing generated artifacts.",
"status": "in_progress",
"dev_type": null,
"scope": null,
"package": null,
"priority": "P2",
"creator": "dingkang",
"assignee": "dingkang",
"createdAt": "2026-08-03",
"completedAt": null,
"branch": null,
"base_branch": "main",
"worktree_path": null,
"commit": null,
"pr_url": null,
"subtasks": [],
"children": [],
"parent": null,
"relatedFiles": [],
"notes": "",
"meta": {}
}

View File

@@ -35,10 +35,7 @@ if _HERE not in sys.path:
from osmassets import catalog # noqa: E402 from osmassets import catalog # noqa: E402
from osmassets.geom import ( # noqa: E402 (needs the sys.path line above) from osmassets.geom import ( # noqa: E402 (needs the sys.path line above)
clip_polygon,
distance_to_ring, distance_to_ring,
feature_in_bounds,
geometry_rings,
point_in_polygon, point_in_polygon,
polygon_area, polygon_area,
sample_polygon_interior, sample_polygon_interior,
@@ -51,7 +48,6 @@ from osmassets.materials import ( # noqa: E402
) )
from osmassets.mesh import ( # noqa: E402 from osmassets.mesh import ( # noqa: E402
MeshBatch, MeshBatch,
add_polyline,
new_collection, new_collection,
) )
from osmassets.osm import Projector, parse_height, parse_osm # noqa: E402 from osmassets.osm import Projector, parse_height, parse_osm # noqa: E402
@@ -59,6 +55,7 @@ from osmassets import building as _building # noqa: E402
from osmassets import fountain as _fountain # noqa: E402 from osmassets import fountain as _fountain # noqa: E402
from osmassets import water as _water # noqa: E402 from osmassets import water as _water # noqa: E402
from osmassets import grass as _grass # noqa: E402 from osmassets import grass as _grass # noqa: E402
from osmassets import roads as _roads # noqa: E402
from osmassets import scrub as _scrub # noqa: E402 from osmassets import scrub as _scrub # noqa: E402
from osmassets import tree as _tree # noqa: E402 from osmassets import tree as _tree # noqa: E402
@@ -145,29 +142,6 @@ def cli_args():
return values return values
def add_geojson_layer(path, layer, projector, collection, material, z):
if not os.path.exists(path):
return 0
with open(path, "r", encoding="utf-8") as handle:
data = json.load(handle)
batch = MeshBatch("Road_" + layer, collection, material)
b = projector.bounds
xmin, ymin = projector.xy((b["min_lon"], b["min_lat"]))
xmax, ymax = projector.xy((b["max_lon"], b["max_lat"]))
count = 0
for feature in data.get("features", []):
if not feature_in_bounds(feature, projector):
continue
for ring in geometry_rings(feature.get("geometry")):
points = [projector.xy(pair) for pair in ring]
points = clip_polygon(points, xmin, xmax, ymin, ymax)
if len(points) >= 3:
batch.add_polygon(points, z)
count += 1
batch.finish()
return count
def add_tree_batch(positions, collection, trunk_material, leaf_material): def add_tree_batch(positions, collection, trunk_material, leaf_material):
trunk = MeshBatch("Tree_Trunks", collection, trunk_material) trunk = MeshBatch("Tree_Trunks", collection, trunk_material)
leaves = MeshBatch("Tree_Crowns", collection, leaf_material) leaves = MeshBatch("Tree_Crowns", collection, leaf_material)
@@ -743,17 +717,13 @@ def build(args):
print("Layer catalog warning:", problem) print("Layer catalog warning:", problem)
for layer in catalog.ROAD_LAYERS: for layer in catalog.ROAD_LAYERS:
layer_id = layer["id"] layer_id = layer["id"]
road_counts[layer_id] = add_geojson_layer( road_counts[layer_id] = _roads.assemble_geojson_layer(
os.path.join(geojson_dir, layer_id + ".geojson"), layer_id, os.path.join(geojson_dir, layer_id + ".geojson"), layer_id,
projector, roads_c, road_mats[layer_id], layer["z"]) projector, roads_c, road_mats[layer_id], layer["z"])
if road_counts.get("road_surface", 0) == 0: if road_counts.get("road_surface", 0) == 0:
for way in ways: _roads.assemble_osm_fallback(
highway = way["tags"].get("highway") ways, projector, roads_c, road_mats["road_surface"])
if highway and len(way["coords"]) >= 2:
width = {"secondary": 7.0, "residential": 5.5, "service": 3.5}.get(highway, 4.0)
add_polyline("OSM_Road_" + str(way["id"]), way["coords"], projector,
roads_c, road_mats["road_surface"], width, 0.03)
trees = [] trees = []
individual_tree_count = 0 individual_tree_count = 0

View File

@@ -0,0 +1,40 @@
"""Road layer assembly from osm2streets GeoJSON and OSM fallback ways."""
import json
import os
from osmassets.geom import clip_polygon, feature_in_bounds, geometry_rings
from osmassets.mesh import MeshBatch, add_polyline
def assemble_geojson_layer(path, layer_id, projector, collection, material, z):
if not os.path.exists(path):
return 0
with open(path, "r", encoding="utf-8") as handle:
data = json.load(handle)
batch = MeshBatch("Road_" + layer_id, collection, material)
b = projector.bounds
xmin, ymin = projector.xy((b["min_lon"], b["min_lat"]))
xmax, ymax = projector.xy((b["max_lon"], b["max_lat"]))
count = 0
for feature in data.get("features", []):
if not feature_in_bounds(feature, projector):
continue
for ring in geometry_rings(feature.get("geometry")):
points = [projector.xy(pair) for pair in ring]
points = clip_polygon(points, xmin, xmax, ymin, ymax)
if len(points) >= 3:
batch.add_polygon(points, z)
count += 1
batch.finish()
return count
def assemble_osm_fallback(ways, projector, collection, material):
for way in ways:
highway = way["tags"].get("highway")
if highway and len(way["coords"]) >= 2:
width = {"secondary": 7.0, "residential": 5.5,
"service": 3.5}.get(highway, 4.0)
add_polyline("OSM_Road_" + str(way["id"]), way["coords"], projector,
collection, material, width, 0.03)

View File

@@ -22,6 +22,11 @@
roof、window bands、bevel 和 building custom properties`generate_scene.py` 继续 roof、window bands、bevel 和 building custom properties`generate_scene.py` 继续
负责 `office_overrides`、材质创建、way 分发、计数和 scene metadata。Blender/Cesium 负责 `office_overrides`、材质创建、way 分发、计数和 scene metadata。Blender/Cesium
parity before/after 在两个样本上均 identicalP2 剩余未拆出部分更新为 `roads` parity before/after 在两个样本上均 identicalP2 剩余未拆出部分更新为 `roads`
- 抽出 roads 要素装配:新增 `blender/osmassets/roads.py`,迁移 osm2streets GeoJSON
road layer mesh assembly 与 OSM highway fallback polyline 装配;`generate_scene.py`
继续负责 `ROAD_LAYERS` 遍历、`catalog.check_layers()` warning、`road_counts`、scene
metadata 和 `SCENE_DONE` JSON。Blender/Cesium parity before/after 在两个样本上均
identicalP2 剩余工作更新为完整 `features/` 注册表设计。
## 2026-07-31远看发黑的真正原因反照率没被提亮 ## 2026-07-31远看发黑的真正原因反照率没被提亮