34 Commits

Author SHA1 Message Date
7f4ebe8fb7 Improve Cesium preview loading state 2026-07-31 18:11:21 +08:00
0def373c09 Extract Cesium preview assets 2026-07-31 16:56:22 +08:00
41a07a78ef Add scrub edge bushes 2026-07-31 16:53:28 +08:00
eb4a0d9b2a fix: 树发黑的真正原因是反照率没提亮,并加饱和度让远看更绿
emissive 提到 0.8 仍然发黑:它乘的是本来就很暗的反照率,0.8 × 深绿约等于
sRGB [0.22, 0.31, 0.11],还是暗的。要提的是反照率本身。

Cesium 截图定位:草亮黄绿、建筑近白、路面浅灰,只有树暗。这棵树图集的叶片
本来就是深绿(绿色系像素均值 sRGB [0.249, 0.35, 0.12]),按真实反照率渲染
是对的,但场景里其他材质都被手工提亮过——EXPORT_TINTS 草 0.72、带肋墙面
0.86,EXPORT_EMISSION_OVERRIDES 建筑 0.18,全是针对 Cesium 偏白的默认光照
调出来的。新资产没调过,是唯一如实渲染的东西,放旁边就显得发黑。

- FOLIAGE_ALBEDO_GAIN = 2.1,在抠图植被 dilate 那一遍顺带乘上。用增益而非
  tint:其他材质是单一表面,而这是同时装着叶片/树皮/果实的图集,往绿色混会
  把树干染绿。缩放保留色相关系,只把整体抬到和邻居一样的曝光。
- FOLIAGE_SATURATION = 1.75,绕各像素自身 Rec.709 亮度做饱和度拉伸。图集本
  来就偏灰(平均饱和度 0.22),远看时 mip 把叶片、树皮、缝隙混在一起会更往
  中性靠。绕亮度拉伸把叶片推绿而基本不动本来中性的部分,也没有绿色 tint 强
  加给树干的色相偏移。叶片 sRGB → [0.262, 0.529, 0.021],饱和度 0.22 → 0.34,
  过曝到纯白的像素占 0.3%。
- FOLIAGE_EMISSION 回调 0.25,它只负责给背光面兜底。
- 新增 triangulate_mesh():export_tangents 打开后刷了 55 行「切向空间只能只算
  三角/四边形」,MeshBatch 建的 OSM 轮廓都是 n-gon。glTF 只有三角形,导出时
  无论如何都会三角化,提前做不改变任何输出三角形(三角数 51719 前后一致),
  切线 49/102 → 102/102,警告归零。

被数据排除的假设:贴图颜色全链路逐位一致;模拟 mip 链可见像素亮度
0.127→0.124;法线贴图抠图区是干净平面法线;GLB 重新导入 Blender 渲染正常。

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 13:23:44 +08:00
afada31469 fix: 树在 Cesium 远看发黑 — 补 emissive 并导出切线
黑色色块修掉后仍存在:近看正常,远看整棵树是暗色块。逐项排除:

- 贴图颜色全链路无偏移,源 4k / 降采样 2k / GLB 内 PNG 的 opaque 均值
  都是 sRGB [0.409, 0.406, 0.362],不是 gamma 问题。
- 模拟 GPU mip 链,可见像素亮度 mip0→mip6 只从 0.127 变到 0.124,
  覆盖率稳定 0.42,不是 mipmap 变暗。
- 法线贴图抠图区域是干净的平面法线 [0.494, 0.512, 0.988],无黑像素。

真实原因是缺 emissive 补偿。预览页 skyBox / skyAtmosphere / sun 全部关闭
且未配置环境贴图,Cesium 只剩很弱的默认球谐环境光,太阳直射不到的面接近
全黑——这正是 EXPORT_EMISSION_OVERRIDES 存在的原因(建筑 0.18,程序化树冠
0.015~0.02)。树冠绝大部分是背光叶片卡片,远看塌成一团暗色,近看能看到
向阳面所以还行。apple 材质不在那张表里。

- 抠图植被把 diffuse 接回 Emission Color,FOLIAGE_EMISSION = 0.22。
  用带贴图的自发光而非平坦颜色:常量会把树干染成叶子绿,而这样每个像素
  下限是自身反照率的一个比例。不增加字节,emissiveTexture 指向 base color
  已在用的同一张图。
- 打开 export_tangents:apple 有法线贴图但图元无 TANGENT,缺失时切线由
  渲染器推导,在双面薄片叶子卡上不可靠。49/102 图元带上切线,顺带修正
  建筑法线贴图,GLB +0.83MB。

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 12:00:06 +08:00
2139990818 feat: 树渲染改用 apple/fattree 模型,修复 Cesium alpha 抠图丢失
tree_style 新增 apple(SpeedTree Red Delicious,4475 tris,alpha 抠图叶片
+ 法线贴图)和 fattree(低面数卡通树,2238 tris,实体几何)。

高度改为按模型自身包围盒归一化到目标高度,树根落在 z=0,不再用魔数;
每棵树按序号做确定性抖动(缩放 ±14%、黄金角偏航、±3° 倾斜),
tree_row 采样高度全部相同,不抖动就是同一棵树盖章 156 次。

移除 polyhaven 样式和 island_tree_01 资产(76MB)+ ingest_tree.py:
该样式 append 的 *_LOD1 并不是完整的树,枝干只有 0.41 单位高,叶片是
挂在原点下方的平面簇,本是给源文件几何节点散布用的碎片。

Cesium 侧修三处:

- make_export_material 把 Alpha 恒定写死为 1.0,叶片卡片整块导出,而
  SpeedTree 图集抠掉的区域是纯黑,在 Cesium 里就是黑色色块。
- Blender 4.2 起 glTF 导出不再读 blend_method(仍可写但已失效,写 CLIP
  读回来是 HASHED),改为从节点树推断 alpha 模式,alpha 直连 BSDF 会落到
  BLEND。新增 materials.link_alpha_clip() 构造导出器识别的
  1 - (alpha < cutoff) 结构,得到 alphaMode=MASK。
- 新增 alpha_dilated_image():把不透明像素颜色向抠图区域外扩 8 圈,
  避免 mipmap 把黑色平均进叶缘。叶缘相邻的纯黑像素 10.3% → 0.6%。

另修两个既有 bug:

- 材质槽在 mesh 上,181 棵树共享一个 datablock,第一棵替换后其余会把结果
  再包一层,产生 Cesium Cesium Cesium... 的材质名;烘焙图缓存按材质名索引,
  每轮再嵌一份同样的贴图。GLB 22.46MB → 20.76MB,materials 270 → 23。
- shrub_02 从 glTF 带进来一个 Math 节点接在 Alpha 上,但其 JPEG 贴图 alpha
  全是 1.0,只判断「连了 Alpha」会误判为抠图材质。

模型资产为第三方素材,已 gitignore,缺失时回落到 natural;
来源见 assets/models/SOURCES.md。

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 11:30:16 +08:00
b35b094ed2 docs: changelog 更新 — P0-P3 refactor + island_tree_01
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-30 09:22:04 +08:00
365b158878 docs: tree.py + ingest_tree.py 补充文档注释
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-30 09:20:40 +08:00
3c6c21ff2f config: nantaizi 默认 treeStyle 切换为 polyhaven
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-30 09:17:16 +08:00
243f0c8ced fix: load_tree_variants() 缺少 return variants 语句
函数在构建 variants 列表后删除了 imported objects 但忘记 return。
Python 隐式返回 None,导致 tree.assemble() 永远进入 'not variants'
分支(但 assemble 里也没有检查 None,只是 silently return 0)。

结果:tree 构建静默失败,场景里一棵树都没有。

181 棵树现在正确出现在 SCENE_DONE 和 GLB 里 (362 objects, 113k tris)。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-30 09:17:07 +08:00
e8aecc7143 refactor: tree.assemble() 接管树构建,_add_polyhaven_trees 移除
把 polyhaven 树的实例化逻辑从 generate_scene.py 移到
osmassets/tree.py::assemble(),与 water/grass/scrub 的要素
注册表模式对齐。

generate_scene.py 现在只有一个 _assemble_building 残留(因为需要
make_prism / add_roof / add_wall_panel 这些 mesh 构建函数)。
后续 P2c 会把 building 也搬出去。

parity: control-1 vs p2b-unify = PARITY OK (两区域)
smoke: tree_style=polyhaven SCENE_DONE (181棵树, exit 0)
tests: 42/42 OK

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-30 09:05:14 +08:00
9ddf951186 chore: grass.py 参数名 add_grass_tufts_fn → add_grass_tufts (更 Pythonic)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-30 09:01:30 +08:00
b2136e708d chore: gitignore island_tree_01 模型资产(55MB + 20MB 贴图)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-30 08:57:35 +08:00
a621ef743b feat: 接入 Poly Haven island_tree_01 真实扫描树模型
tree_style 新增 'polyhaven' 选项,用 wm.append 从 vendored .blend
加载 LOD1 网格(4 种 branch+leaf 组合),通过实例化放置场景中的树。

每棵树约 625 tris / 2 meshes (branch+leaf),比旧 procedural tree
(~400 tris) 多了约 60%,但换来真实树形和 PBR 贴图,树叶有 alpha
透明度,在 Cesium 远距离比纯几何球体更可读。

纹理从 ~/Downloads 手动下载(Poly Haven CDN 的 API URL 不可用),
存放在 assets/models/polyhaven/island_tree_01/textures/(不入 git)。

默认 tree_style 仍为 'natural',parity 保持不变。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-30 08:57:24 +08:00
24e02e2041 refactor: 抽 osmassets 包,要素注册表,材质单一定义源 (P0-P3)
generate_scene.py 从 1271 → 816 行 (-455)

P0: 纯函数搬家
- osmassets/osm.py: parse_osm / Projector / parse_height
- osmassets/geom.py: clip_polygon / point_in_polygon / distance_to_ring / sample_tree_row 等
- blender/tests/test_pure.py: 42 个 unittest (脱离 bpy 运行)

P1: 单一定义源
- osmassets/catalog.py: ROAD_LAYERS + MATERIALS (含 cesium 导出参数)
- 对接 osm2streets_scene_style.json 做图层一致性 warning
- 干掉 road_mats / layer_z / 材质参数三份副本

P2: 要素注册表
- osmassets/{water,grass,scrub}.py: 每个要素一个 assemble() 函数
- build() 中的 if/elif 链收缩为注册表调用
- 计数器集中到 counts 字典

P3: 材质契约化
- catalog.py 扩展 CESIUM_EXPORT 段 (tint/metallic/emission)
- 标记已发现的死条目 Office White Metal Facade (四表各一组)

校验:
- parity.js + scene_digest.py + glb-digest.js 三位一体
- control-1 vs p0/p1/p2a/p2b/p3-counts: 两区域全 PARITY OK
- 42 个纯 Python 测试全部通过

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-29 17:59:32 +08:00
23ae63bc2a feat: 草坪新增一些草 2026-07-29 14:46:29 +08:00
59628cba82 feat: add reimport stage for hand-edited GeoPackages
QGIS 手工修正后的回导流程此前是 README 里的三段 shell:一个硬编码
area-id 和 ogr2ogr 绝对路径的 for 循环、一段内联 node heredoc、再加一次
npm run build。改为一个 reimport 阶段:

  npm run build -- --config config/areas/<area-id>.json \
    --stages reimport,blender,cesium

scripts/reimport-gpkg.js 先把全部图层导出到临时目录并逐个校验,全部通过
才写回 osm2streets_web_out/。ogr2ogr 对不存在的图层退出码非 0 但仍会留下
0 字节文件,原先逐图层 mv 会静默用空图层覆盖好数据。

intermediates 与 reimport 同时指定直接报错——前者用 OSM 重建 GeoPackage,
正好抹掉后者要读回的手工修改。reimport 不含在 all 中。

同时新增 scripts/lib/scene-layers.js 作为 9 个渲染图层的唯一定义源。此前
该表在合并场景、场景样式 JSON、生成的 QGIS 工程、README 手工流程中各有
一份副本,改一处漏其余会导致图层叠放顺序出错并流入 Blender/Cesium。

验证:同一 OSM 输入下,改动前后 intermediates 产物逐字节一致
(scene_style.json、.qgz 符号定义、9 个图层几何与属性);reimport 对
未修改图层无损回导。

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-28 17:21:59 +08:00
2b18d0eb2b fix: synthesize T-junction sidewalk caps 2026-07-28 10:20:47 +08:00
03ebb7c0ed fix: backfill missing sidewalk corners 2026-07-28 10:00:57 +08:00
96c9baf540 feat: osm2streets上游箭头修复 2026-07-28 09:08:58 +08:00
694f18826a Add multi-vehicle cruise selection preview 2026-07-27 15:55:29 +08:00
4fc9397cec Fix cruise vehicle model orientation 2026-07-27 15:07:48 +08:00
633f1ee364 Restore lightweight cruise car and keep lane offset 2026-07-27 14:53:40 +08:00
d93212b685 Use clean ToyCar model and lane-offset cruise route 2026-07-27 14:31:56 +08:00
9561ad8530 Use lightweight glTF car for cruise preview 2026-07-27 14:19:06 +08:00
513ef62ab8 Allow zoom adjustment in vehicle follow mode 2026-07-27 14:12:09 +08:00
235314af1a Replace tracked vehicle follow with chase camera 2026-07-27 14:06:46 +08:00
d872ca6f32 Make vehicle following opt-in 2026-07-27 14:00:49 +08:00
f1ddedf889 Add experimental Cesium vehicle cruise preview 2026-07-27 13:52:43 +08:00
633c39e5e1 Remove Nantaizi building render overrides 2026-07-27 13:41:02 +08:00
5919684888 Treat mislabeled Nantaizi buildings as offices 2026-07-27 13:28:55 +08:00
a5c3ad3cf3 Generate Cesium preview pages from area builds 2026-07-27 13:06:57 +08:00
61e4820a4c Refactor pipeline around area asset builds 2026-07-27 12:56:13 +08:00
59a22ea697 Use lightweight natural tree rendering 2026-07-27 12:36:55 +08:00
46 changed files with 6727 additions and 197402 deletions

3
.gitignore vendored
View File

@@ -1,3 +1,6 @@
.DS_Store
node_modules/
outputs/
__pycache__/
assets/models/speedtree/
assets/models/lyrog/

223
README.md
View File

@@ -1,14 +1,26 @@
# OSM GIS Pipeline
# OSM Asset Pipeline
把 OSM XML 转为 QGIS 工程osm2streets 风格道路)和可选的 Blender 3D 场景
单个园区/片区 OSM XML 转为可消费的 Blender 场景和 Cesium GLB。osm2streets GeoJSON、GeoPackage、QGIS 工程和预览图都是中间资产,用来提供道路几何、调试标线效果,以及给 Blender/Cesium 生成提供输入
## 目标产物
每个区域默认输出到 `outputs/<area-id>/`
- `<area-id>.blend`Blender 场景,包含道路、建筑、水体、植被等
- `<area-id>.png`Blender 预览渲染
- `<area-id>.glb`Cesium 可加载的 3D 模型
- `<area-id>.json`Cesium 放置元数据和示例代码
- `<area-id>-cesium-preview.html`Cesium 本地预览页
- `osm2streets_web_out/`osm2streets GeoJSON 中间层
- `<area-id>.gpkg` / `<area-id>.qgz` / `<area-id>-preview.png`QGIS 调试资产
## 环境
需要:
- macOS QGIS默认 `/Applications/QGIS.app`
- Blender默认 `/Applications/Blender.app`
- Node.js / npm
- Blender可选用于 3D 场景生成)
首次使用:
@@ -17,93 +29,122 @@ cd /Users/que01/osm2streets-qgis-workflow
npm install
```
## QGIS 管线(强依赖 osm2streets
## 主流程
使用默认配置
默认构建南台子湖创新谷样例
```bash
cd /Users/que01/osm2streets-qgis-workflow
npm run build
```
使用指定配置:
指定区域配置:
```bash
node scripts/build-osm2streets-qgis.js --config /path/to/config.json
npm run build:area -- --config config/areas/hanyang-block.json
```
命令行参数可以覆盖配置文件
只跑部分阶段
```bash
node scripts/build-osm2streets-qgis.js \
--input /path/to/osm.xml \
--out-dir /path/to/out \
--gpkg /path/to/output.gpkg \
--project /path/to/output.qgz \
--preview /path/to/output_preview.png
npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json --stages intermediates
npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json --stages blender
npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json --stages cesium
npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json --stages preview
npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json --stages reimport
```
创建新区域配置可以从模板复制:
`intermediates` 会生成 osm2streets GeoJSON、GeoPackage、QGIS 工程和 QGIS 预览图。`blender` 使用 OSM 和 osm2streets GeoJSON 生成 `.blend`/`.png``cesium``.blend` 导出 `.glb`/`.json`,并生成 Cesium 预览 HTML。`preview` 只在已有 `.glb/.json` 时补生成 HTML。`reimport` 把手工编辑过的 GeoPackage 回导为 GeoJSON不含在 `all` 里,详见 [QGIS 手工修正工作流](#qgis-手工修正工作流)。
## 区域配置
新区域从模板复制:
```bash
cp config/examples/template.json config/my-area.json
cp config/examples/template.json config/areas/my-area.json
```
### 配置项
核心配置:
```json
{
"qgisApp": "/Applications/QGIS.app",
"id": "my-area",
"input": "/absolute/path/to/input.osm",
"outDir": "/absolute/path/to/osm2streets_web_out",
"gpkg": "/absolute/path/to/output.gpkg",
"project": "/absolute/path/to/output.qgz",
"preview": "/absolute/path/to/output_preview.png",
"arrowScale": 0.8,
"clipPad": 0.002,
"canvasPad": 0.001,
"previewPad": 0.0007,
"canvasExtent": null,
"previewExtent": null,
"layerPrefix": "osm2streets",
"osm2streets": {
"debug_each_step": false,
"dual_carriageway_experiment": false,
"sidepath_zipping_experiment": false,
"inferred_sidewalks": true,
"osm2lanes": true
"outputRoot": "/absolute/path/to/outputs",
"qgisApp": "/Applications/QGIS.app",
"blenderApp": "/Applications/Blender.app",
"stages": {
"intermediates": true,
"blender": true,
"cesium": true
},
"qgis": {
"arrowScale": 0.8,
"arrowMergeTriangles": true,
"arrowOutlineSimplifyMeters": 0.05,
"intersectionCornerSourceMaxDimensionMeters": 2.6,
"clipPad": 0.002,
"canvasPad": 0.001,
"previewPad": 0.0007,
"canvasExtent": null,
"previewExtent": null,
"layerPrefix": "osm2streets"
},
"blender": {
"treeStyle": "natural",
"officeOverrides": ""
}
}
```
- `input`OSM XML 输入文件路径。
- `outDir`osm2streets GeoJSON 中间产物输出目录。
- `gpkg` / `project` / `preview`:最终 GeoPackage、QGIS 工程、预览 PNG 路径。
- `layerPrefix`QGIS 图层显示名称前缀(如 `"osm2streets"``"osm2streets road surface"`)。
- `arrowScale`:方向箭头几何缩放。
- `osm2streets`osm2streets 引擎选项。
QGIS road-layer knobs:
### 输出图层
- `arrowScale`: scales osm2streets lane-arrow polygons before export.
- `arrowMergeTriangles`: merges each osm2streets lane-arrow triangle mesh into one valid polygon. This preserves the original arrow shape and turn direction while removing renderer gaps along shared triangle edges.
- `arrowOutlineSimplifyMeters`: removes sub-decimeter kinks from the merged arrow exterior. The default `0.05` removes the two malformed tail vertices without changing the arrow head; the remaining tail edge is aligned perpendicular to the shaft.
- `intersectionCornerSourceMaxDimensionMeters`: keeps only small osm2streets `sidewalk corner` polygons. Large intersection-marking polygons are not treated as sidewalk because they can cover the drivable junction.
GeoPackage 内会生成
`scripts/build-area.js` 会按 `id` 自动推导默认输出路径。确实需要定制时,可以增加 `outputs` 覆盖
- `road_surface`
- `intersection_surface`
- `sidewalks`
- `sidewalk_corners`
- `lane_separators`
- `center_lines`
- `vehicle_stop_lines`
- `lane_arrows_webscale`
- `crosswalks`
```json
{
"outputs": {
"areaDir": "/absolute/path/to/custom-area",
"blend": "/absolute/path/to/custom.blend",
"glb": "/absolute/path/to/custom.glb",
"cesiumPreview": "/absolute/path/to/custom-preview.html"
}
}
```
QGIS 工程绘制顺序:路面在最底,车道线、斑马线、停止线和箭头在上。
预览 Cesium 页面时,需要在输出目录启动 HTTP 服务,避免浏览器拦截本地文件请求:
## Blender 3D 场景(可选)
```bash
cd outputs/my-area
python3 -m http.server 8765
```
osm2streets 道路几何为可选输入;不提供时回退到 OSM highway 折线
然后打开 `http://localhost:8765/my-area-cesium-preview.html`
生成场景:
## 实验:车辆巡航
`preview``cesium` 阶段会额外生成 `<area-id>-vehicle-route.json``<area-id>-vehicle-car.gltf`。路线文件从 OSM bounds 内的可行驶 `highway` way 提取道路中心线,并向右偏移约 1.3 米作为车辆行驶线避免车辆压道路中心线。Cesium 预览页会加载多条道路段并显示多辆实验车辆循环巡航;`Vehicle` 下拉框决定 `Follow` 跟随哪一辆车。
这是用于验证高精度巡航可用性的预览层功能,不会改变 Blender/GLB 主资产本身。车辆模型是无 logo 的轻量预览模型,生成在输出目录中。
## 已沉淀区域
- `config/areas/nantaizi-lake-innovation-valley.json`
- `config/areas/hanyang-block.json`
## 低层命令
通常优先使用 `npm run build:area`。如果只想调试旧 QGIS/osm2streets 阶段,可以直接运行:
```bash
npm run build:qgis -- --config config/hanyang-block.json
```
Blender 低层命令:
```bash
/Applications/Blender.app/Contents/MacOS/Blender \
@@ -112,70 +153,52 @@ osm2streets 道路几何为可选输入;不提供时回退到 OSM highway 折
--osm "/path/to/input.osm" \
--geojson "/path/to/osm2streets_web_out" \
--output "/path/to/scene.blend" \
--render "/path/to/preview.png"
--render "/path/to/preview.png" \
--tree-style natural
```
`--geojson` 为可选参数。使用 `--office-overrides` 指定应渲染为办公楼的 OSM way ID逗号分隔
导出为 Cesium GLB
Cesium GLB 低层导出:
```bash
/Applications/Blender.app/Contents/MacOS/Blender \
--background \
--python blender/export_cesium.py -- \
--blend "/path/to/scene.blend" \
--glb "/path/to/scene_cesium.glb" \
--metadata "/path/to/scene_cesium.json"
--glb "/path/to/scene.glb" \
--metadata "/path/to/scene.json"
```
详见 [blender/README.md](blender/README.md)。
## 运行示例
## QGIS 手工修正工作流
以下命令已在本机验证通过。
如果已经在现成的 `.qgz` 项目里直接编辑了 `gpkg` 图层,不要再重跑 `intermediates`,否则会把手工修改覆盖掉。推荐流程是:
QGIS 管线(使用汉阳区区块 OSM 数据):
1. 在 QGIS 中打开 `outputs/<area-id>/<area-id>.qgz`
2. 直接编辑项目内关联的 `gpkg` 图层并保存
3.`reimport` 回导 GeoJSON 并重建场景,再接 `blender,cesium`
南台子湖创新谷当前可直接使用下面这条命令:
```bash
node scripts/build-osm2streets-qgis.js \
--config config/hanyang-block.json
npm run build -- --config config/areas/nantaizi-lake-innovation-valley.json --stages reimport,blender,cesium
```
Blender 场景(含 osm2streets 道路几何)
`reimport` 阶段(`scripts/reimport-gpkg.js`)做两件事
```bash
/Applications/Blender.app/Contents/MacOS/Blender \
--background --factory-startup \
--python blender/generate_scene.py -- \
--osm "/Users/que01/Desktop/南台子湖创新谷OSM.osm" \
--geojson "/Users/que01/osm2streets-qgis-workflow/outputs/nantaizi-lake-innovation-valley/osm2streets_web_out" \
--output "/tmp/scene.blend" \
--render "/tmp/preview.png"
```
-`<area-id>.gpkg` 里的 9 个图层逐个导出到 `osm2streets_web_out/<layer>.geojson`
-`scripts/lib/scene-layers.js` 的图层表重建 `osm2streets_scene.geojson`(写入 `render_layer` / `z_index`)和 `osm2streets_scene_style.json`
Blender 场景(纯 OSM无 osm2streets 道路)
说明
```bash
/Applications/Blender.app/Contents/MacOS/Blender \
--background --factory-startup \
--python blender/generate_scene.py -- \
--osm "/Users/que01/Desktop/南台子湖创新谷OSM.osm" \
--output "/tmp/scene-nogeojson.blend" \
--render "/tmp/preview-nogeojson.png"
```
Cesium GLB 导出:
```bash
/Applications/Blender.app/Contents/MacOS/Blender \
--background \
--python blender/export_cesium.py -- \
--blend "/tmp/scene.blend" \
--glb "/tmp/scene.glb" \
--metadata "/tmp/scene.json"
```
- 这套流程假设你的手工修改已经保存在 `outputs/<area-id>/<area-id>.gpkg`
- 所有图层先导出到临时目录并校验通过后才写回 `osm2streets_web_out/`;任一图层缺失或导出结果不是合法 FeatureCollection整批都不落盘`ogr2ogr` 遇到不存在的图层会留下 0 字节文件,直接覆盖会静默损坏数据)
- `intermediates``reimport` 互斥,同时指定会直接报错:前者用 OSM 重建 `gpkg`,正好会抹掉后者要读回的手工修改
- `blender,cesium` 阶段读取的是 `osm2streets_web_out/*.geojson`,不是直接读取 `gpkg`
- 如果 Blender 当前环境不稳定,先确认 `geojson` 已完成回导,再单独排查 Blender 本身
- 增删图层或调整 `z_index` 只需改 `scripts/lib/scene-layers.js`构建、场景合并、场景样式、QGIS 工程会一并同步
## 文档
- [docs/input-cases.md](docs/input-cases.md) 已验证的 OSM 输入案例
- [docs/changelog.md](docs/changelog.md) 变更记录
- [docs/input-cases.md](docs/input-cases.md) - 已验证的 OSM 输入案例
- [docs/changelog.md](docs/changelog.md) - 变更记录

47
assets/models/SOURCES.md Normal file
View File

@@ -0,0 +1,47 @@
# 树模型资产来源
`assets/models/` 下的树模型是第三方素材,**已 gitignore**,不随仓库分发。
干净 checkout 缺这些文件时 `tree.assemble()` 返回 0`generate_scene.py`
自动回落到 `natural` 样式,构建不会失败。
放置路径见 `blender/osmassets/tree.py` 顶部的常量。
## apple`--tree-style apple`
- 名称RedDeliciousAppleSpeedTree 工具导出)
- 路径:`assets/models/speedtree/apple_low/`
- `RedDeliciousApple.obj` — 4475 tris单网格
- `RedDeliciousApple.mtl` — 已重写,见文件内注释
- `textures/apple_color_2k.png` — 由原始 4096² PNG 降采样,**必须保留 alpha 通道**
57% 的像素是抠掉的叶片卡片
- `textures/apple_normal_2k.png` — 同上降采样
原始素材另有 `.fbx` / `.cgf` / `.st` / `.tif` 以及 `_SS` / `_Subsurface` 贴图,
管线不用,未收录。
降采样命令(原始 4k 贴图 16MB + 20MB降到 2k 后 4.3MB + 4.7MB
```bash
sips -Z 2048 -s format png <原始>_Color.png --out textures/apple_color_2k.png
sips -Z 2048 -s format png <原始>_Normal.png --out textures/apple_normal_2k.png
```
## fattree`--tree-style fattree`
- 名称fat tree作者署名 Lyrog付费素材
- 路径:`assets/models/lyrog/fattree/`
- `fattree.blend` — 取其中名为 `fattree` 的对象2238 tris
- `textures/fat_tree.png` — 原文件名带空格(`fat tree.png`),已改为下划线
源 .blend 里的贴图路径是 `//fat tree.png`,只在原始目录下能解析;
运行时不依赖它,`tree.py` 会丢掉源材质并按上面的路径重建一个 Principled 材质。
该 .blend 里另有两个 `plant` 对象(灌木),管线未使用。
## 已移除
`polyhaven/island_tree_01`76MB已于 2026-07-31 删除,连同 `--tree-style polyhaven`
`blender/tools/ingest_tree.py`。原因见 `docs/changelog.md`:该文件的 `*_LOD1`
对象不是完整的树,而是给几何节点散布用的枝叶碎片。
若要重新取用Poly Haven 上是 CC0https://polyhaven.com/a/island_tree_01
`polyhaven/shrub_02` 仍在使用(草丛散布),不受影响。

Binary file not shown.

View File

@@ -1,16 +0,0 @@
newmtl None
Ka 1.000000 1.000000 1.000000
Kd 1.000000 1.000000 1.000000
Ks 0.000000 0.000000 0.000000
d 1.000000
illum 1
map_Kd HazelnutBark.png
newmtl None_Hazelnut.png
Ka 1.000000 1.000000 1.000000
Kd 1.000000 1.000000 1.000000
Ks 0.000000 0.000000 0.000000
d 1.000000
illum 1
map_Kd HazelnutLeaves.png
map_d HazelnutLeaves.png

File diff suppressed because it is too large Load Diff

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.2 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 5.8 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 187 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

View File

@@ -0,0 +1,437 @@
{
"asset": {
"generator": "Khronos glTF Blender I/O v3.5.30",
"version": "2.0"
},
"scene": 0,
"scenes": [
{
"name": "Scene",
"nodes": [
0,
1,
2,
3
]
}
],
"nodes": [
{
"mesh": 0,
"name": "shrub_02_a",
"rotation": [
0.12589912116527557,
-0.6947680711746216,
0.006097283214330673,
0.708102822303772
],
"translation": [
-1.5478131771087646,
-0.005240630358457565,
-0.02073405683040619
]
},
{
"mesh": 1,
"name": "shrub_02_b",
"rotation": [
0.08358937501907349,
-0.746630847454071,
0.26227879524230957,
0.6056113243103027
],
"translation": [
0.0007123738760128617,
-0.006340932101011276,
-0.01680799387395382
]
},
{
"mesh": 2,
"name": "shrub_02_c",
"rotation": [
0.12648986279964447,
-0.7129753232002258,
0.3943961560726166,
0.5657898783683777
],
"translation": [
1.5559492111206055,
-0.005484403111040592,
-0.006101916544139385
]
},
{
"mesh": 3,
"name": "shrub_02_d",
"rotation": [
0.047377295792102814,
-0.7383221983909607,
-0.300414502620697,
0.60198575258255
],
"translation": [
3.258664131164551,
-0.004767145495861769,
-0.005447957664728165
]
}
],
"materials": [
{
"alphaCutoff": 0.5,
"alphaMode": "MASK",
"doubleSided": true,
"name": "shrub_02",
"normalTexture": {
"index": 0
},
"pbrMetallicRoughness": {
"baseColorTexture": {
"index": 1
},
"metallicFactor": 0,
"metallicRoughnessTexture": {
"index": 2
}
}
}
],
"meshes": [
{
"name": "Plane.024",
"primitives": [
{
"attributes": {
"POSITION": 0,
"TEXCOORD_0": 1,
"NORMAL": 2
},
"indices": 3,
"material": 0
}
]
},
{
"name": "Plane.029",
"primitives": [
{
"attributes": {
"POSITION": 4,
"TEXCOORD_0": 5,
"NORMAL": 6
},
"indices": 7,
"material": 0
}
]
},
{
"name": "Plane.041",
"primitives": [
{
"attributes": {
"POSITION": 8,
"TEXCOORD_0": 9,
"NORMAL": 10
},
"indices": 11,
"material": 0
}
]
},
{
"name": "Plane.036",
"primitives": [
{
"attributes": {
"POSITION": 12,
"TEXCOORD_0": 13,
"NORMAL": 14
},
"indices": 15,
"material": 0
}
]
}
],
"textures": [
{
"sampler": 0,
"source": 0
},
{
"sampler": 0,
"source": 1
},
{
"sampler": 0,
"source": 2
}
],
"images": [
{
"mimeType": "image/jpeg",
"name": "shrub_02_nor_gl",
"uri": "textures/shrub_02_nor_gl_1k.jpg"
},
{
"mimeType": "image/jpeg",
"name": "shrub_02_diff",
"uri": "textures/shrub_02_diff_1k.jpg"
},
{
"mimeType": "image/jpeg",
"name": "shrub_02_rough",
"uri": "textures/shrub_02_arm_1k.jpg"
}
],
"accessors": [
{
"bufferView": 0,
"componentType": 5126,
"count": 5125,
"max": [
0.6156559586524963,
1.6821810007095337,
0.42684367299079895
],
"min": [
-0.9439988136291504,
-0.03012576885521412,
-0.863244354724884
],
"type": "VEC3"
},
{
"bufferView": 1,
"componentType": 5126,
"count": 5125,
"type": "VEC2"
},
{
"bufferView": 2,
"componentType": 5126,
"count": 5125,
"type": "VEC3"
},
{
"bufferView": 3,
"componentType": 5123,
"count": 22770,
"type": "SCALAR"
},
{
"bufferView": 4,
"componentType": 5126,
"count": 3600,
"max": [
0.9647624492645264,
1.0844767093658447,
0.30848613381385803
],
"min": [
-0.4038015305995941,
-0.10226154327392578,
-1.0773800611495972
],
"type": "VEC3"
},
{
"bufferView": 5,
"componentType": 5126,
"count": 3600,
"type": "VEC2"
},
{
"bufferView": 6,
"componentType": 5126,
"count": 3600,
"type": "VEC3"
},
{
"bufferView": 7,
"componentType": 5123,
"count": 15726,
"type": "SCALAR"
},
{
"bufferView": 8,
"componentType": 5126,
"count": 6346,
"max": [
1.287977695465088,
0.8719967007637024,
0.24143657088279724
],
"min": [
-0.8910231590270996,
-0.2947101294994354,
-1.22649085521698
],
"type": "VEC3"
},
{
"bufferView": 9,
"componentType": 5126,
"count": 6346,
"type": "VEC2"
},
{
"bufferView": 10,
"componentType": 5126,
"count": 6346,
"type": "VEC3"
},
{
"bufferView": 11,
"componentType": 5123,
"count": 27702,
"type": "SCALAR"
},
{
"bufferView": 12,
"componentType": 5126,
"count": 3528,
"max": [
0.22877177596092224,
1.153072476387024,
0.5660104751586914
],
"min": [
-1.051486849784851,
-0.09016040712594986,
-0.41910019516944885
],
"type": "VEC3"
},
{
"bufferView": 13,
"componentType": 5126,
"count": 3528,
"type": "VEC2"
},
{
"bufferView": 14,
"componentType": 5126,
"count": 3528,
"type": "VEC3"
},
{
"bufferView": 15,
"componentType": 5123,
"count": 15564,
"type": "SCALAR"
}
],
"bufferViews": [
{
"buffer": 0,
"byteLength": 61500,
"byteOffset": 0,
"target": 34962
},
{
"buffer": 0,
"byteLength": 41000,
"byteOffset": 61500,
"target": 34962
},
{
"buffer": 0,
"byteLength": 61500,
"byteOffset": 102500,
"target": 34962
},
{
"buffer": 0,
"byteLength": 45540,
"byteOffset": 164000,
"target": 34963
},
{
"buffer": 0,
"byteLength": 43200,
"byteOffset": 209540,
"target": 34962
},
{
"buffer": 0,
"byteLength": 28800,
"byteOffset": 252740,
"target": 34962
},
{
"buffer": 0,
"byteLength": 43200,
"byteOffset": 281540,
"target": 34962
},
{
"buffer": 0,
"byteLength": 31452,
"byteOffset": 324740,
"target": 34963
},
{
"buffer": 0,
"byteLength": 76152,
"byteOffset": 356192,
"target": 34962
},
{
"buffer": 0,
"byteLength": 50768,
"byteOffset": 432344,
"target": 34962
},
{
"buffer": 0,
"byteLength": 76152,
"byteOffset": 483112,
"target": 34962
},
{
"buffer": 0,
"byteLength": 55404,
"byteOffset": 559264,
"target": 34963
},
{
"buffer": 0,
"byteLength": 42336,
"byteOffset": 614668,
"target": 34962
},
{
"buffer": 0,
"byteLength": 28224,
"byteOffset": 657004,
"target": 34962
},
{
"buffer": 0,
"byteLength": 42336,
"byteOffset": 685228,
"target": 34962
},
{
"buffer": 0,
"byteLength": 31128,
"byteOffset": 727564,
"target": 34963
}
],
"samplers": [
{
"magFilter": 9729,
"minFilter": 9987
}
],
"buffers": [
{
"byteLength": 758692,
"uri": "shrub_02.bin"
}
]
}

Binary file not shown.

After

Width:  |  Height:  |  Size: 306 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 386 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 479 KiB

View File

@@ -2,6 +2,14 @@
该生成器将 OSM 建筑、水体、植被标签,以及可选的 osm2streets GeoJSON 道路图层,组合为 Blender 3D 场景。
项目主入口是根目录的区域管线:
```bash
npm run build:area -- --config config/areas/nantaizi-lake-innovation-valley.json --stages blender,cesium
```
下面的命令适合单独调试 Blender 脚本。
## 生成场景
```bash
@@ -11,10 +19,19 @@
--osm "/path/to/input.osm" \
--geojson "/path/to/osm2streets_web_out" \
--output "/path/to/output.blend" \
--render "/path/to/preview.png"
--render "/path/to/preview.png" \
--tree-style natural
```
`--geojson` 为可选参数。不提供时,道路使用简单的 OSM highway 折线,而非详细 osm2streets 几何。
`--geojson` 为可选参数。不提供时,道路使用简单的 OSM highway 折线,而非详细 osm2streets 几何。实际资产生成建议先跑 `intermediates` 阶段,为 Blender 提供 osm2streets 道路面、标线、斑马线和箭头。
`--tree-style` 可选 `apple``fattree``natural``procedural`
- `apple`拟真——SpeedTree Red Delicious4.5k tris叶片是 alpha 抠图卡片,带法线贴图。
- `fattree`卡通——低面数卡通树2.2k tris纯实体几何无抠图。
- `natural`——低面数树干 + 多层不规则树冠,不依赖外部模型。
- `procedural`——最轻,球形树冠。
`apple``fattree` 依赖 `assets/models/` 下的第三方模型(已 gitignore。模型缺失时自动回落到 `natural`,干净 checkout 仍可构建;实际使用的样式记录在场景的 `tree_style_used` 属性里。
使用 `--office-overrides` 指定一组 OSM way ID逗号分隔这些建筑将渲染为办公楼风格即使其 OSM 标签为 `building=industrial`
@@ -25,7 +42,7 @@
### 说明
- OSM `bounds` 元素定义场景范围(排除远处的地铁等关系成员)
- `natural=tree` 节点 → 独立树木(程序化或模型树冠)
- `natural=tree` 节点 → 独立树木(默认低面数自然树冠)
- `natural=tree_row` 路径 → 沿路径均匀分布的树木
- `landuse=grass` → 绿色地面
- `natural=scrub` → 低矮灌木丛
@@ -48,6 +65,7 @@
GLB 使用以 OSM bounds 中心为原点的局部 ENU 坐标系X 东Y 北Z 上)。
使用配套的 JSON 元数据文件将模型放置到 Cesium 中。
区域管线的 `cesium` 阶段还会在输出目录生成 `<area-id>-cesium-preview.html`
导出脚本会:
- 应用网格修改器并创建 UV
@@ -63,4 +81,4 @@ GLB 使用以 OSM bounds 中心为原点的局部 ENU 坐标系X 东Y 北
```bash
cd /path/to/output
python3 -m http.server 8765
```
```

View File

@@ -16,9 +16,60 @@ import sys
import bpy
import numpy as np
# --factory-startup does not put the script's own directory on sys.path, so the
# osmassets package next to this file is not importable without this.
_HERE = os.path.dirname(os.path.abspath(__file__))
if _HERE not in sys.path:
sys.path.insert(0, _HERE)
from osmassets.materials import link_alpha_clip # noqa: E402
# Marks a material this exporter produced, so a second pass over an instanced
# mesh's shared slots can recognise its own output and leave it alone.
EXPORT_PREFIX = "Cesium "
# Fraction of its own albedo a cut-out foliage material emits, to keep the
# shadowed side of a crown off Cesium's near-black ambient floor. Kept well
# under the 0.18 the buildings use: a tree still has to read as lit from one
# side, it just must not go to black.
FOLIAGE_EMISSION = 0.25
# Multiplier on a cut-out foliage albedo before export.
#
# This is the knob that actually controls how dark the trees read, and it
# exists because the apple atlas is genuinely dark: its green texels average
# sRGB (0.249, 0.35, 0.12), a deep forest green, and the bark is darker still.
# Rendered at true albedo that is correct — but nothing else in this scene is
# at true albedo. Every other material goes through EXPORT_TINTS (grass mixes
# 72% toward a bright green, the ribbed facade 86% toward white) and
# EXPORT_EMISSION_OVERRIDES (0.18 on the buildings), all hand-tuned against
# Cesium's washed-out default lighting. A new asset dropped in untuned is the
# one thing rendering honestly, and next to the rest it reads as black.
#
# A gain rather than a tint, because a tint is what the other materials use and
# it is wrong here: they are single-surface, this is an atlas holding leaves,
# bark and fruit at once, and mixing it toward green would turn the trunk
# green. Scaling preserves the hue relationships and just lifts the whole
# thing into the same exposure as its neighbours.
FOLIAGE_ALBEDO_GAIN = 2.1
# Saturation multiplier applied with the gain, around each texel's own
# luminance. The gain alone lifts the crown to the right brightness but leaves
# it reading grey-green at distance: this atlas is desaturated to begin with
# (mean saturation 0.22), and mip-averaging a crown mixes leaves with bark and
# sky-gaps, pulling it further toward neutral exactly when the tree gets small.
#
# Scaling the distance from luminance pushes the leaves green without touching
# what is already neutral much, and without the hue shift a green tint would
# force on the trunk — bark just becomes a warmer brown, which it should be.
FOLIAGE_SATURATION = 1.75
EXPORT_TINTS = {
"Grass": ((0.12, 0.48, 0.08), 0.72),
"Tree Crown Dark": ((0.06, 0.22, 0.05), 0.18),
"Tree Crown Light": ((0.16, 0.42, 0.09), 0.16),
"Scrub Ground Cover": ((0.08, 0.28, 0.07), 0.28),
"Office White Plaster Facade": ((0.92, 0.94, 0.92), 0.38),
"Office White Metal Facade": ((0.92, 0.94, 0.92), 0.68),
"Office Light Flat Roof": ((0.82, 0.86, 0.88), 0.35),
@@ -33,12 +84,18 @@ EXPORT_METALLIC_OVERRIDES = {
}
EXPORT_BASE_COLOR_OVERRIDES = {
"Tree Crown": (0.11, 0.34, 0.075),
"Tree Crown Dark": (0.065, 0.24, 0.055),
"Tree Crown Light": (0.14, 0.40, 0.085),
"Office White Plaster Facade": (0.93, 0.94, 0.91),
"Office White Metal Facade": (0.93, 0.94, 0.91),
"Office Light Flat Roof": (0.88, 0.90, 0.88),
}
EXPORT_EMISSION_OVERRIDES = {
"Tree Crown": ((0.04, 0.11, 0.035), 0.02),
"Tree Crown Dark": ((0.025, 0.07, 0.02), 0.015),
"Tree Crown Light": ((0.045, 0.12, 0.03), 0.015),
"Office White Plaster Facade": ((0.93, 0.94, 0.91), 0.18),
"Office White Metal Facade": ((0.93, 0.94, 0.91), 0.18),
"Office Light Flat Roof": ((0.88, 0.90, 0.88), 0.14),
@@ -85,10 +142,17 @@ def source_color(material):
return color
def source_principled_value(material, input_name, fallback):
def principled_bsdf(material):
if not material.use_nodes:
return fallback
node = material.node_tree.nodes.get("Principled BSDF")
return None
for node in material.node_tree.nodes:
if node.type == "BSDF_PRINCIPLED":
return node
return None
def source_principled_value(material, input_name, fallback):
node = principled_bsdf(material)
if not node or input_name not in node.inputs:
return fallback
return node.inputs[input_name].default_value
@@ -101,6 +165,48 @@ def source_texture_scale(material):
return (1.0, 1.0, 1.0)
def source_alpha_clipped(material):
"""Whether the source material carries its silhouette in a texture alpha.
Two conditions, because either alone gives a wrong answer. A link into the
Principled Alpha input is not enough: shrub_02 arrives from glTF with a
Math node wired there even though its JPEG diffuse is opaque, and taking
that at face value re-encodes an opaque texture as a PNG and makes Cesium
alpha-test 270 tufts for nothing. An alpha channel alone is not enough
either: fat_tree.png is RGBA with every texel at 1.0.
So ask both — the author wired alpha, and the texture actually cuts.
"""
node = principled_bsdf(material)
if not node or "Alpha" not in node.inputs:
return False
if not node.inputs["Alpha"].links:
return False
diffuse = image_for(material, want_normal=False)
return diffuse is not None and image_has_cutout(diffuse)
_CUTOUT_CACHE = {}
def image_has_cutout(image, threshold=0.5):
"""Whether any of the image's texels are transparent enough to be cut.
A full pass over the pixel buffer, so memoise it — the exporter asks once
per material and several materials can share one texture.
"""
if image.name in _CUTOUT_CACHE:
return _CUTOUT_CACHE[image.name]
width, height = image.size
result = False
if width and height:
alpha = np.empty(width * height * 4, dtype=np.float32)
image.pixels.foreach_get(alpha)
result = bool((alpha[3::4] < threshold).any())
_CUTOUT_CACHE[image.name] = result
return result
def tinted_image(source, name, tint, factor):
existing = bpy.data.images.get(name)
if existing:
@@ -125,7 +231,71 @@ def cesium_tinted_image(material, source):
return source
color, factor = tint
safe_name = material.name.replace(" ", "_")
return tinted_image(source, f"Cesium {safe_name} Baked", color, factor)
return tinted_image(
source, f"{EXPORT_PREFIX}{safe_name} Baked", color, factor)
def alpha_dilated_image(source, name, threshold=0.5, passes=8, gain=1.0,
saturation=1.0):
"""Flood the opaque colour outward underneath the cut-out, and lift it.
SpeedTree writes pure black wherever a leaf card is cut away — 97% of the
apple atlas's transparent area is exactly (0, 0, 0). An alpha mask hides
that at full resolution, but Cesium mip-maps the texture and every mip
level averages those black texels into the leaf edges, so the crown grows a
dark fringe that thickens with distance. Blender's preview renders at mip
0 and never shows it, which is why this only surfaces in the viewer.
Replacing the colour under the cut-out with its nearest opaque neighbours
leaves no black to bleed. Alpha is copied through untouched, so the
silhouette is byte-for-byte what it was.
`gain` and `saturation` grade the result into the same exposure and colour
as the rest of the scene — see FOLIAGE_ALBEDO_GAIN and FOLIAGE_SATURATION.
Both are applied after the flood so the filled border keeps matching the
leaves it was copied from, and the result is clipped at 1.0.
"""
existing = bpy.data.images.get(name)
if existing:
return existing
width, height = source.size
pixels = np.empty(width * height * 4, dtype=np.float32)
source.pixels.foreach_get(pixels)
rgba = pixels.reshape((height, width, 4))
rgb = rgba[..., :3].copy()
filled = rgba[..., 3] >= threshold
# Each pass pushes the colour one texel further out, so `passes` is how
# many mip levels' worth of filter footprint gets covered.
for _ in range(passes):
if filled.all():
break
weight = filled[..., None].astype(np.float32)
total = np.zeros_like(rgb)
count = np.zeros((height, width, 1), dtype=np.float32)
for shift, axis in ((1, 0), (-1, 0), (1, 1), (-1, 1)):
total += np.roll(rgb * weight, shift, axis=axis)
count += np.roll(weight, shift, axis=axis)
edge = (~filled) & (count[..., 0] > 0)
rgb[edge] = total[edge] / count[edge]
filled = filled | edge
if saturation != 1.0:
# Rec.709 luminance, so the push is around perceived brightness rather
# than the channel average.
luma = rgb @ np.asarray([0.2126, 0.7152, 0.0722], dtype=np.float32)
rgb = luma[..., None] + (rgb - luma[..., None]) * saturation
if gain != 1.0 or saturation != 1.0:
rgb = np.clip(rgb * gain, 0.0, 1.0)
dilated = rgba.copy()
dilated[..., :3] = rgb
result = bpy.data.images.new(name, width=width, height=height, alpha=True)
result.file_format = "PNG"
result.colorspace_settings.name = "sRGB"
result.pixels.foreach_set(dilated.ravel())
result.pack()
return result
def tree_crown_image():
@@ -144,8 +314,8 @@ def tree_crown_image():
np.sin((x * 89.0 + y * 67.0) * np.pi) * 0.07
)
noise = np.clip(0.5 + noise, 0.0, 1.0)[..., None]
dark = np.asarray((0.035, 0.16, 0.045), dtype=np.float32)
light = np.asarray((0.12, 0.42, 0.13), dtype=np.float32)
dark = np.asarray((0.04, 0.17, 0.04), dtype=np.float32)
light = np.asarray((0.17, 0.46, 0.115), dtype=np.float32)
rgb = dark + (light - dark) * noise
rgba = np.concatenate(
(rgb, np.ones((size, size, 1), dtype=np.float32)), axis=2)
@@ -159,7 +329,7 @@ def tree_crown_image():
def make_export_material(material):
result = material.copy()
result.name = "Cesium " + material.name
result.name = EXPORT_PREFIX + material.name
result.use_nodes = True
nodes = result.node_tree.nodes
links = result.node_tree.links
@@ -189,10 +359,18 @@ def make_export_material(material):
diffuse = image_for(material, want_normal=False)
normal = image_for(material, want_normal=True)
# Foliage that carries its silhouette in the texture's alpha has to keep
# that channel; every other material is flattened to opaque below.
alpha_clipped = source_alpha_clipped(material)
if material.name == "Tree Crown":
diffuse = tree_crown_image()
else:
diffuse = cesium_tinted_image(material, diffuse)
if alpha_clipped and diffuse is not None:
safe_name = material.name.replace(" ", "_")
diffuse = alpha_dilated_image(
diffuse, f"{EXPORT_PREFIX}{safe_name} Dilated",
gain=FOLIAGE_ALBEDO_GAIN, saturation=FOLIAGE_SATURATION)
if material.name in EXPORT_BASE_COLOR_OVERRIDES:
diffuse = None
normal = None
@@ -205,6 +383,7 @@ def make_export_material(material):
mapping.inputs["Scale"].default_value = source_texture_scale(material)
links.new(texcoord.outputs["UV"], mapping.inputs["Vector"])
diffuse_node = None
if diffuse:
image = nodes.new("ShaderNodeTexImage")
image.location = (-200, 80)
@@ -212,6 +391,7 @@ def make_export_material(material):
image.extension = "REPEAT"
links.new(mapping.outputs["Vector"], image.inputs["Vector"])
links.new(image.outputs["Color"], bsdf.inputs["Base Color"])
diffuse_node = image
if normal:
normal_tex = nodes.new("ShaderNodeTexImage")
@@ -226,7 +406,33 @@ def make_export_material(material):
links.new(normal_tex.outputs["Color"], normal_map.inputs["Color"])
links.new(normal_map.outputs["Normal"], bsdf.inputs["Normal"])
if "Alpha" in bsdf.inputs:
if alpha_clipped and diffuse_node is not None:
# A leaf crown is a handful of quads whose shape lives entirely in this
# channel. Pinning Alpha to 1.0 — which is what the rest of the scene
# wants — exports those quads whole, and the cut-away regions of a
# SpeedTree atlas are black, so Cesium draws black slabs.
link_alpha_clip(result, diffuse_node.outputs["Alpha"], bsdf,
cutoff=material.alpha_threshold)
# Lift the crown out of Cesium's ambient. The preview configures no
# environment map, so anything the sun does not hit directly falls to a
# weak default spherical-harmonic term — which is why every other
# material here carries an EXPORT_EMISSION_OVERRIDES entry. A crown is
# mostly self-shadowed leaf cards facing away from the sun, so at
# distance it collapses into one dark mass while a sunlit close-up
# still reads fine.
#
# Feed the diffuse back in as the emissive texture rather than using a
# flat colour: a constant would wash the bark with leaf green, whereas
# this floors every texel at a fraction of its own albedo. It costs no
# extra bytes — the exporter points emissiveTexture at the image the
# base colour already uses.
if "Emission Color" in bsdf.inputs:
links.new(diffuse_node.outputs["Color"], bsdf.inputs["Emission Color"])
elif "Emission" in bsdf.inputs:
links.new(diffuse_node.outputs["Color"], bsdf.inputs["Emission"])
if "Emission Strength" in bsdf.inputs:
bsdf.inputs["Emission Strength"].default_value = FOLIAGE_EMISSION
elif "Alpha" in bsdf.inputs:
bsdf.inputs["Alpha"].default_value = 1.0
return result
@@ -234,6 +440,11 @@ def make_export_material(material):
def unwrap_mesh(obj):
if obj.type != "MESH" or not obj.data.polygons:
return
# Imported assets ship authored UVs that map onto their own texture atlas;
# smart_project would scramble the leaves. Only the procedurally built
# meshes arrive without a UV layer, so that is the reliable discriminator.
if obj.data.uv_layers:
return
bpy.ops.object.select_all(action="DESELECT")
obj.select_set(True)
bpy.context.view_layer.objects.active = obj
@@ -258,6 +469,37 @@ def apply_mesh_modifiers(obj):
pass
def triangulate_mesh(obj):
"""Split n-gons into triangles ahead of the exporter.
glTF has no n-gons, so the exporter triangulates on the way out regardless
— doing it here does not change a single output triangle. What it changes
is tangents: Blender can only build a tangent basis on tris and quads, and
every footprint this pipeline extrudes from OSM is an n-gon, so with
export_tangents on each one logged "切向空间只能只算三角/四边形" and shipped
without a basis. Triangulating first turns ~55 failures into tangents.
Skipped for meshes that are already triangles, which covers the instanced
props — those share one datablock across hundreds of objects and
modifier_apply refuses to touch multi-user data.
"""
if obj.type != "MESH" or not obj.data.polygons:
return
if all(len(polygon.vertices) <= 3 for polygon in obj.data.polygons):
return
bpy.ops.object.select_all(action="DESELECT")
obj.select_set(True)
bpy.context.view_layer.objects.active = obj
modifier = obj.modifiers.new("ExportTriangulate", "TRIANGULATE")
modifier.min_vertices = 4
try:
bpy.ops.object.modifier_apply(modifier=modifier.name)
except RuntimeError:
# Multi-user data. The exporter still triangulates it, we just lose the
# tangent basis for that mesh.
obj.modifiers.remove(modifier)
def export(args):
if not os.path.exists(args["blend"]):
raise FileNotFoundError(args["blend"])
@@ -265,6 +507,7 @@ def export(args):
material_map = {}
meshes = []
unwrapped = set()
for obj in bpy.context.scene.objects:
if obj.type != "MESH":
continue
@@ -274,11 +517,24 @@ def export(args):
continue
meshes.append(obj)
apply_mesh_modifiers(obj)
unwrap_mesh(obj)
# Hundreds of grass tufts share four mesh datablocks; unwrapping and
# triangulating are properties of the mesh, so once per datablock.
if obj.data.name not in unwrapped:
unwrapped.add(obj.data.name)
triangulate_mesh(obj)
unwrap_mesh(obj)
for slot in obj.material_slots:
if not slot.material:
continue
source = slot.material
# Instanced props share one mesh datablock, and material slots live
# on the mesh, so the first tree already swapped in the export
# material for all 181 of them. Without this the next instance
# wraps that result again — "Cesium Cesium Cesium ..." — and since
# the baked-image cache is keyed by material name, every round
# embeds another multi-megabyte copy of the same texture.
if source.name.startswith(EXPORT_PREFIX):
continue
if source.name not in material_map:
material_map[source.name] = make_export_material(source)
slot.material = material_map[source.name]
@@ -298,6 +554,11 @@ def export(args):
export_normals=True,
export_materials="EXPORT",
export_image_format="AUTO",
# The foliage materials carry a normal map, and glTF leaves tangent
# derivation to the renderer when TANGENT is absent. On thin
# double-sided leaf cards that derivation is unreliable, so ship real
# tangents — it costs four floats a vertex on meshes this small.
export_tangents=True,
export_extras=True,
export_cameras=False,
export_lights=False,
@@ -315,6 +576,13 @@ def export(args):
center_lon = center_lat = 0.0
metadata = {
"asset": os.path.basename(args["glb"]),
"assets": [{
"id": "main",
"label": "Scene",
"type": "model",
"url": os.path.basename(args["glb"]),
"enabled": True,
}],
"coordinate_system": "local ENU meters (X east, Y north, Z up)",
"heading_correction_degrees": -90.0,
"anchor": {"longitude": center_lon, "latitude": center_lat, "height": 0.35},
@@ -353,4 +621,4 @@ def export(args):
if __name__ == "__main__":
export(cli_args())
export(cli_args())

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,12 @@
"""Reusable pieces of the OSM → Blender/Cesium asset pipeline.
The package is split by dependency, not by feature:
- `osm` and `geom` are pure Python. They import no `bpy` and can be run and
tested with a plain interpreter (`python3 -m unittest discover blender/tests`).
- everything else may touch `bpy` and only runs inside Blender.
Keeping that line sharp is what makes the geometry testable at all; before the
split it was interleaved with scene construction and could only be exercised by
rendering a whole area.
"""

View File

@@ -0,0 +1,195 @@
"""Single source of truth for the scene's road layers and materials.
Before this module the same facts lived in several places at once: the nine
osm2streets layers had their draw order in `scripts/lib/scene-layers.js`, their
Blender heights in a `layer_z` dict, and their colours in a `road_mats` dict —
three copies across two languages, kept in sync by hand. Everything the scene
builder needs is now declared here, once.
Two deliberate non-goals:
- The colours here are NOT derived from `scene-layers.js`. That file's `fill`
values are QGIS sRGB hex for a 2D debug map; these are linear Blender base
colours for a 3D scene, and the two were tuned separately. `check_layers`
cross-checks the layer *set and order* — the part that must agree — and
leaves the palettes alone.
- Order is load-bearing. Material creation order fixes the material indices in
the exported GLB, and layer order fixes mesh creation order, so both lists
are sequences, not dicts, and appending is the only safe edit.
"""
import json
import os
# Draw order, bottom first. `z` is the Blender height in metres that keeps the
# markings above the asphalt without z-fighting; `id` matches the GeoJSON file
# stem written by the intermediates stage.
ROAD_LAYERS = [
{"id": "road_surface", "material": "Road Asphalt",
"color": (0.055, 0.065, 0.070), "z": 0.03},
{"id": "intersection_surface", "material": "Intersection Asphalt",
"color": (0.065, 0.075, 0.080), "z": 0.035},
{"id": "sidewalks", "material": "Sidewalk",
"color": (0.49, 0.51, 0.49), "z": 0.065},
{"id": "sidewalk_corners", "material": "Sidewalk Corner",
"color": (0.49, 0.51, 0.49), "z": 0.067},
{"id": "lane_separators", "material": "Lane Separator",
"color": (0.85, 0.84, 0.72), "z": 0.090},
{"id": "center_lines", "material": "Center Line",
"color": (0.94, 0.58, 0.06), "z": 0.092},
{"id": "crosswalks", "material": "Crosswalk",
"color": (0.95, 0.94, 0.82), "z": 0.094},
{"id": "vehicle_stop_lines", "material": "Stop Line",
"color": (0.95, 0.94, 0.82), "z": 0.096},
{"id": "lane_arrows_webscale", "material": "Lane Arrow",
"color": (0.95, 0.94, 0.82), "z": 0.098},
]
SCENE_STYLE_FILE = "osm2streets_scene_style.json"
# Material specs. `kind` selects the builder:
# solid — flat base colour
# textured — Poly Haven diffuse + normal, optionally tinted
# `procedural` adds noise-driven base colour and bump on top of a solid.
MATERIALS = {
"ground": {"kind": "solid", "name": "Ground", "color": (0.27, 0.32, 0.24)},
"water": {"kind": "solid", "name": "Lake Water", "color": (0.035, 0.22, 0.30),
"roughness": 0.18, "metallic": 0.05},
"grass": {"kind": "textured", "name": "Grass",
"diffuse": "leafy_grass_diff_1k.jpg",
"normal": "leafy_grass_nor_gl_1k.jpg",
"roughness": 0.92, "scale": 7.0,
"tint": (0.12, 0.48, 0.08), "tint_factor": 0.72},
"scrub": {"kind": "textured", "name": "Scrub Ground Cover",
"diffuse": "leafy_grass_diff_1k.jpg",
"normal": "leafy_grass_nor_gl_1k.jpg",
"roughness": 0.96, "scale": 15.0,
"tint": (0.085, 0.30, 0.065), "tint_factor": 0.46},
"fountain_stone": {"kind": "solid", "name": "Fountain Stone",
"color": (0.42, 0.45, 0.43), "roughness": 0.72},
"fountain_water": {"kind": "solid", "name": "Fountain Water",
"color": (0.03, 0.32, 0.42), "roughness": 0.16,
"metallic": 0.05},
"fountain_spray": {"kind": "solid", "name": "Fountain Spray",
"color": (0.20, 0.70, 0.78), "roughness": 0.12,
"metallic": 0.02},
"building_default": {"kind": "textured", "name": "Office White Plaster Facade",
"diffuse": "white_plaster_02_diff_1k.jpg",
"normal": "white_plaster_02_nor_gl_1k.jpg",
"roughness": 0.82, "scale": 4.2, "metallic": 0.0,
"tint": (0.92, 0.94, 0.92), "tint_factor": 0.38},
"building_industrial": {"kind": "textured",
"name": "Industrial White Ribbed Facade",
"diffuse": "corrugated_iron_03_diff_1k.jpg",
"normal": "corrugated_iron_03_nor_gl_1k.jpg",
"roughness": 0.56, "scale": 2.4, "metallic": 0.16,
"tint": (0.86, 0.92, 0.94), "tint_factor": 0.68},
"building_office_roof": {"kind": "textured", "name": "Office Light Flat Roof",
"diffuse": "concrete_floor_02_diff_1k.jpg",
"normal": "concrete_floor_02_bump_1k.jpg",
"roughness": 0.84, "scale": 5.0,
"normal_is_bump": True,
"tint": (0.82, 0.86, 0.88), "tint_factor": 0.35},
"building_industrial_roof": {"kind": "textured",
"name": "Factory Blue Metal Roof",
"diffuse": "blue_metal_plate_diff_1k.jpg",
"normal": "blue_metal_plate_nor_gl_1k.jpg",
"roughness": 0.48, "scale": 3.4, "metallic": 0.28,
"tint": (0.03, 0.42, 0.78), "tint_factor": 0.45},
"building_glass": {"kind": "solid", "name": "Office Blue Gray Glass",
"color": (0.12, 0.20, 0.24), "roughness": 0.22,
"metallic": 0.10},
"building_factory_glass": {"kind": "solid", "name": "Factory Dark Windows",
"color": (0.10, 0.14, 0.15), "roughness": 0.28,
"metallic": 0.08},
"tree_trunk": {"kind": "textured", "name": "Tree Trunk",
"diffuse": "bark_brown_01_diff_1k.jpg",
"normal": "bark_brown_01_nor_gl_1k.jpg",
"roughness": 0.92, "scale": 5.0},
"tree_crown_dark": {"kind": "solid", "name": "Tree Crown Dark",
"color": (0.065, 0.25, 0.055), "roughness": 0.90,
"procedural": {"colors": ((0.035, 0.14, 0.035),
(0.12, 0.36, 0.08)),
"scale": 3.2, "detail": 3.8,
"bump_strength": 0.08},
"cesium": {"tint": ((0.06, 0.22, 0.05), 0.18)}},
"tree_crown_light": {"kind": "solid", "name": "Tree Crown Light",
"color": (0.13, 0.42, 0.09), "roughness": 0.88,
"procedural": {"colors": ((0.07, 0.25, 0.05),
(0.22, 0.56, 0.13)),
"scale": 3.6, "detail": 3.4,
"bump_strength": 0.07},
"cesium": {"tint": ((0.16, 0.42, 0.09), 0.16)}},
"tree_crown": {"kind": "solid", "name": "Tree Crown",
"color": (0.10, 0.36, 0.08), "roughness": 0.88,
"procedural": {"colors": ((0.04, 0.18, 0.04),
(0.18, 0.50, 0.12)),
"scale": 2.8, "detail": 3.2,
"bump_strength": 0.10},
"cesium": {"tint": None,
"base_color": (0.11, 0.34, 0.075),
"emission": ((0.04, 0.11, 0.035), 0.02)}},
}
# Cesium-specific overrides that don't have a home in the material system yet:
# metallic overrides (flat values, not materials) and emission overrides for
# colours that export_cesium.py hand-tuned separately.
CESIUM_EXPORT = {
"metallic_overrides": {
"Office White Plaster Facade": 0.0,
"Industrial White Ribbed Facade": 0.08,
},
"emission_overrides": {
"Office White Plaster Facade": ((0.93, 0.94, 0.91), 0.18),
"Office Light Flat Roof": ((0.88, 0.90, 0.88), 0.14),
"Industrial White Ribbed Facade": ((0.90, 0.93, 0.91), 0.18),
"Factory Blue Metal Roof": ((0.08, 0.50, 0.88), 0.12),
},
}
def road_material_specs():
"""Road layer materials as MATERIALS-shaped specs, in draw order."""
return [{"kind": "solid", "name": layer["material"], "color": layer["color"]}
for layer in ROAD_LAYERS]
def check_layers(geojson_dir):
"""Warn when the intermediates stage and this catalog disagree on layers.
The style JSON is written next to the GeoJSON by the intermediates and
reimport stages. A layer added on the JS side but not here would be
silently dropped from the 3D scene, which is exactly the kind of drift the
single-source-of-truth split is meant to make loud. Warn rather than fail:
a stale or absent output directory should not block a rebuild.
"""
style_path = os.path.join(geojson_dir or "", SCENE_STYLE_FILE)
if not geojson_dir or not os.path.exists(style_path):
return []
try:
with open(style_path, "r", encoding="utf-8") as handle:
style = json.load(handle)
except (OSError, ValueError) as error:
return ["Could not read %s: %s" % (style_path, error)]
upstream = [entry.get("id") for entry in style.get("layers", [])]
local = [layer["id"] for layer in ROAD_LAYERS]
problems = []
for missing in [i for i in upstream if i not in local]:
problems.append(
"layer '%s' exists in %s but not in catalog.ROAD_LAYERS "
"(it will not reach the 3D scene)" % (missing, SCENE_STYLE_FILE))
for extra in [i for i in local if i not in upstream]:
problems.append(
"layer '%s' is in catalog.ROAD_LAYERS but not in %s "
"(no GeoJSON will be produced for it)" % (extra, SCENE_STYLE_FILE))
if not problems and upstream != local:
problems.append(
"layer draw order differs: %s produces %s, catalog stacks %s"
% (SCENE_STYLE_FILE, upstream, local))
return problems

241
blender/osmassets/geom.py Normal file
View File

@@ -0,0 +1,241 @@
"""Planar geometry helpers for the OSM → asset pipeline.
Pure Python: no `bpy`, so this runs and tests outside Blender. All functions
work in projected metres (see `osmassets.osm.Projector`) unless the name says
otherwise; `geometry_rings` and `feature_in_bounds` take raw GeoJSON and are the
two exceptions, operating on lon/lat.
Rings are lists of (x, y) tuples. A repeated closing point is tolerated
everywhere but never required.
"""
import math
def geometry_rings(geometry):
"""Exterior rings of a GeoJSON Polygon/MultiPolygon; holes are dropped."""
if not geometry:
return []
kind = geometry.get("type")
coordinates = geometry.get("coordinates", [])
if kind == "Polygon":
return coordinates[:1]
if kind == "MultiPolygon":
return [polygon[0] for polygon in coordinates if polygon]
return []
def feature_in_bounds(feature, projector):
"""True when any coordinate of the feature falls inside the padded bounds."""
def walk(value):
if isinstance(value, list) and value and isinstance(value[0], (int, float)):
return projector.inside(value)
return any(walk(v) for v in value) if isinstance(value, list) else False
return walk(feature.get("geometry", {}).get("coordinates", []))
def clip_polygon(ring, xmin, xmax, ymin, ymax):
"""Sutherland-Hodgman clip of a ring against an axis-aligned box."""
if len(ring) < 3:
return []
def clip_edge(points, inside, intersection):
if not points:
return []
result = []
previous = points[-1]
previous_inside = inside(previous)
for current in points:
current_inside = inside(current)
if current_inside != previous_inside:
result.append(intersection(previous, current))
if current_inside:
result.append(current)
previous = current
previous_inside = current_inside
return result
ring = clip_edge(
ring, lambda p: p[0] >= xmin,
lambda a, b: (xmin, a[1] + (b[1] - a[1]) * (xmin - a[0]) /
(b[0] - a[0]) if b[0] != a[0] else a[1]))
ring = clip_edge(
ring, lambda p: p[0] <= xmax,
lambda a, b: (xmax, a[1] + (b[1] - a[1]) * (xmax - a[0]) /
(b[0] - a[0]) if b[0] != a[0] else a[1]))
ring = clip_edge(
ring, lambda p: p[1] >= ymin,
lambda a, b: (a[0] + (b[0] - a[0]) * (ymin - a[1]) /
(b[1] - a[1]) if b[1] != a[1] else a[0], ymin))
ring = clip_edge(
ring, lambda p: p[1] <= ymax,
lambda a, b: (a[0] + (b[0] - a[0]) * (ymax - a[1]) /
(b[1] - a[1]) if b[1] != a[1] else a[0], ymax))
return ring
def sample_tree_row(points, spacing, height):
"""Evenly space (x, y, height) samples along a polyline.
The trailing point is appended only when the last regular sample stops well
short of it, so a row does not end in a double-planted tree.
"""
if len(points) < 2:
return []
samples = [(points[0][0], points[0][1], height)]
distance_until_next = spacing
for start, end in zip(points, points[1:]):
dx = end[0] - start[0]
dy = end[1] - start[1]
segment_length = math.hypot(dx, dy)
if segment_length == 0:
continue
while distance_until_next <= segment_length:
ratio = distance_until_next / segment_length
samples.append((start[0] + dx * ratio, start[1] + dy * ratio, height))
distance_until_next += spacing
distance_until_next -= segment_length
last = points[-1]
if math.hypot(samples[-1][0] - last[0], samples[-1][1] - last[1]) > spacing * 0.45:
samples.append((last[0], last[1], height))
return samples
def polygon_area(ring):
"""Unsigned shoelace area; 0.0 for degenerate rings."""
return abs(signed_polygon_area(ring))
def signed_polygon_area(ring):
"""Signed shoelace area; positive for counter-clockwise rings."""
if len(ring) < 3:
return 0.0
area = 0.0
for (x1, y1), (x2, y2) in zip(ring, ring[1:] + ring[:1]):
area += x1 * y2 - x2 * y1
return area * 0.5
def sample_ring_boundary(ring, spacing, inset=0.0, max_samples=None):
"""Evenly sample a closed ring's boundary.
Returns (x, y, angle, index) samples. ``angle`` follows the local edge
direction, and ``inset`` moves the sample toward the polygon interior.
"""
if len(ring) > 1 and ring[0] == ring[-1]:
ring = ring[:-1]
if len(ring) < 3 or spacing <= 0.0:
return []
edges = []
perimeter = 0.0
winding = signed_polygon_area(ring)
for index, (start, end) in enumerate(zip(ring, ring[1:] + ring[:1])):
dx = end[0] - start[0]
dy = end[1] - start[1]
length = math.hypot(dx, dy)
if length <= 1e-9:
continue
ux = dx / length
uy = dy / length
# Counter-clockwise rings have their interior on the left side of each
# edge; clockwise rings have it on the right.
inward = (-uy, ux) if winding >= 0.0 else (uy, -ux)
edges.append((perimeter, start, ux, uy, length, inward, index))
perimeter += length
if not edges:
return []
count = max(1, int(perimeter / spacing))
if max_samples:
count = min(count, max_samples)
step = perimeter / count
samples = []
edge_cursor = 0
for sample_index in range(count):
target = (sample_index + 0.5) * step
while edge_cursor + 1 < len(edges) and (
edges[edge_cursor][0] + edges[edge_cursor][4] < target
):
edge_cursor += 1
edge_start, start, ux, uy, length, inward, _ = edges[edge_cursor]
along = max(0.0, min(length, target - edge_start))
x = start[0] + ux * along
y = start[1] + uy * along
sx = x + inward[0] * inset
sy = y + inward[1] * inset
if inset > 0.0 and not point_in_polygon((sx, sy), ring):
sx, sy = x, y
samples.append((sx, sy, math.atan2(uy, ux), sample_index))
return samples
def sample_polygon_interior(ring, spacing, edge_clearance=0.0, max_samples=None,
seed=0):
"""Jittered interior samples for sparse planting inside a polygon."""
if len(ring) > 1 and ring[0] == ring[-1]:
ring = ring[:-1]
if len(ring) < 3 or spacing <= 0.0 or polygon_area(ring) <= 1e-9:
return []
xmin = min(x for x, _ in ring)
xmax = max(x for x, _ in ring)
ymin = min(y for _, y in ring)
ymax = max(y for _, y in ring)
cols = max(1, int(math.ceil((xmax - xmin) / spacing)))
rows = max(1, int(math.ceil((ymax - ymin) / spacing)))
samples = []
for col in range(cols):
for row in range(rows):
sample_seed = ((col + 1) * 73856093) ^ ((row + 1) * 19349663) ^ seed
jx = ((sample_seed * 0.61803398875) % 1.0 - 0.5) * spacing * 0.7
jy = ((sample_seed * 0.41421356237) % 1.0 - 0.5) * spacing * 0.7
x = xmin + (col + 0.5) * spacing + jx
y = ymin + (row + 0.5) * spacing + jy
if not point_in_polygon((x, y), ring):
continue
if edge_clearance > 0.0 and distance_to_ring((x, y), ring) < edge_clearance:
continue
samples.append((x, y, sample_seed))
if max_samples and len(samples) > max_samples:
samples.sort(key=lambda item: (item[2] * 0.754877666) % 1.0)
samples = samples[:max_samples]
return samples
def point_in_polygon(point, ring):
x, y = point
inside = False
j = len(ring) - 1
for i, (xi, yi) in enumerate(ring):
xj, yj = ring[j]
crosses = ((yi > y) != (yj > y))
if crosses:
x_at_y = (xj - xi) * (y - yi) / (yj - yi) + xi
if x < x_at_y:
inside = not inside
j = i
return inside
def distance_to_ring(point, ring):
"""Shortest distance from a point to the ring's edges (not its interior)."""
px, py = point
best = float("inf")
count = len(ring)
for index in range(count):
ax, ay = ring[index]
bx, by = ring[(index + 1) % count]
dx = bx - ax
dy = by - ay
length_sq = dx * dx + dy * dy
if length_sq <= 1e-9:
distance = math.hypot(px - ax, py - ay)
else:
t = ((px - ax) * dx + (py - ay) * dy) / length_sq
t = max(0.0, min(1.0, t))
distance = math.hypot(px - (ax + t * dx), py - (ay + t * dy))
if distance < best:
best = distance
return best

View File

@@ -0,0 +1,22 @@
"""Grass feature assembly (`landuse=grass`) with optional tuft scattering."""
from osmassets.geom import clip_polygon
from osmassets.mesh import MeshBatch
def assemble(ring, way_id, scene_xmin, scene_xmax, scene_ymin, scene_ymax,
green_c, grass_mat, tuft_variants, add_grass_tufts):
ring = clip_polygon(ring, scene_xmin, scene_xmax, scene_ymin, scene_ymax)
name = "Grass_" + str(way_id)
focus = list(ring)
if len(ring) < 3:
return 0, 0, focus
batch = MeshBatch(name, green_c, grass_mat)
batch.add_polygon(ring, 0.015)
obj = batch.finish()
tufts = 0
if tuft_variants:
tufts = add_grass_tufts(name, ring, tuft_variants, green_c)
if obj:
obj["grass_tufts"] = tufts
return 1, tufts, focus

View File

@@ -0,0 +1,218 @@
"""Blender material construction.
Requires `bpy`; only runs inside Blender. The catalog (`osmassets.catalog`)
declares *what* a material is, this module builds it — that split is what keeps
the catalog importable by plain Python, and by anything else that wants to read
the scene's material definitions without launching Blender.
"""
import os
import bpy
TEXTURE_ROOT = os.path.abspath(os.path.join(
os.path.dirname(os.path.abspath(__file__)), "..", "..",
"assets", "textures", "polyhaven"
))
def principled_bsdf(material):
if not material.use_nodes:
return None
for node in material.node_tree.nodes:
if node.type == "BSDF_PRINCIPLED":
return node
return None
def make_material(name, color, roughness=0.8, metallic=0.0):
material = bpy.data.materials.get(name) or bpy.data.materials.new(name)
material.diffuse_color = (*color, 1.0)
material.use_nodes = True
bsdf = principled_bsdf(material)
if bsdf:
bsdf.inputs["Base Color"].default_value = (*color, 1.0)
bsdf.inputs["Roughness"].default_value = roughness
bsdf.inputs["Metallic"].default_value = metallic
return material
def add_procedural_surface(material, colors, scale=2.0, detail=2.0, bump_strength=0.08,
object_space=False):
nodes = material.node_tree.nodes
links = material.node_tree.links
bsdf = principled_bsdf(material)
if not bsdf:
return
noise = nodes.new("ShaderNodeTexNoise")
noise.inputs["Scale"].default_value = scale
noise.inputs["Detail"].default_value = detail
noise.inputs["Roughness"].default_value = 0.65
texcoord = nodes.new("ShaderNodeTexCoord")
ramp = nodes.new("ShaderNodeValToRGB")
ramp.color_ramp.elements[0].color = (*colors[0], 1.0)
ramp.color_ramp.elements[1].color = (*colors[1], 1.0)
bump = nodes.new("ShaderNodeBump")
bump.inputs["Strength"].default_value = bump_strength
bump.inputs["Distance"].default_value = 0.12
# "Generated" normalises across the object bounding box, so on a mesh that
# spans the whole scene the noise stretches to tens of metres and vanishes.
# Object space keeps the scale in metres, which is what foliage needs.
source = "Object" if object_space else "Generated"
links.new(texcoord.outputs[source], noise.inputs["Vector"])
links.new(noise.outputs["Fac"], ramp.inputs["Fac"])
links.new(ramp.outputs["Color"], bsdf.inputs["Base Color"])
links.new(noise.outputs["Fac"], bump.inputs["Height"])
links.new(bump.outputs["Normal"], bsdf.inputs["Normal"])
def tint_base_color(material, tint, factor):
"""Mix an existing material's base colour toward `tint`.
Imported assets arrive with their own diffuse texture wired up. Rather than
replacing it — which throws away the leaf detail — this splices a mix node
in front of the Base Color input so the texture survives at (1 - factor).
"""
if factor <= 0.0 or not material.use_nodes:
return
bsdf = principled_bsdf(material)
if not bsdf:
return
nodes = material.node_tree.nodes
links = material.node_tree.links
base = bsdf.inputs["Base Color"]
tint_node = nodes.new("ShaderNodeRGB")
tint_node.outputs["Color"].default_value = (*tint, 1.0)
mix = nodes.new("ShaderNodeMixRGB")
mix.blend_type = "MIX"
mix.inputs["Fac"].default_value = factor
if base.is_linked:
# Capture the upstream socket before relinking; Blender drops the old
# link as soon as the input takes a new one.
links.new(base.links[0].from_socket, mix.inputs[1])
else:
mix.inputs[1].default_value = base.default_value
links.new(tint_node.outputs["Color"], mix.inputs[2])
links.new(mix.outputs["Color"], base)
def make_textured_material(name, diffuse_file, normal_file, roughness,
scale, normal_is_bump=False, metallic=0.0,
tint=None, tint_factor=0.0):
diffuse_path = os.path.join(TEXTURE_ROOT, diffuse_file)
normal_path = os.path.join(TEXTURE_ROOT, normal_file)
if not os.path.exists(diffuse_path) or not os.path.exists(normal_path):
return make_material(name, (0.5, 0.5, 0.5), roughness, metallic)
material = make_material(name, (0.5, 0.5, 0.5), roughness, metallic)
nodes = material.node_tree.nodes
links = material.node_tree.links
bsdf = principled_bsdf(material)
if not bsdf:
return material
texcoord = nodes.new("ShaderNodeTexCoord")
mapping = nodes.new("ShaderNodeMapping")
mapping.inputs["Scale"].default_value = (scale, scale, scale)
diffuse = nodes.new("ShaderNodeTexImage")
diffuse.image = bpy.data.images.load(diffuse_path, check_existing=True)
diffuse.extension = "REPEAT"
normal = nodes.new("ShaderNodeTexImage")
normal.image = bpy.data.images.load(normal_path, check_existing=True)
normal.image.colorspace_settings.name = "Non-Color"
normal.extension = "REPEAT"
links.new(texcoord.outputs["Generated"], mapping.inputs["Vector"])
links.new(mapping.outputs["Vector"], diffuse.inputs["Vector"])
links.new(mapping.outputs["Vector"], normal.inputs["Vector"])
if tint and tint_factor > 0.0:
tint_node = nodes.new("ShaderNodeRGB")
tint_node.outputs["Color"].default_value = (*tint, 1.0)
mix = nodes.new("ShaderNodeMixRGB")
mix.blend_type = "MIX"
mix.inputs["Fac"].default_value = tint_factor
links.new(diffuse.outputs["Color"], mix.inputs[1])
links.new(tint_node.outputs["Color"], mix.inputs[2])
links.new(mix.outputs["Color"], bsdf.inputs["Base Color"])
else:
links.new(diffuse.outputs["Color"], bsdf.inputs["Base Color"])
if normal_is_bump:
bump = nodes.new("ShaderNodeBump")
bump.inputs["Strength"].default_value = 0.22
bump.inputs["Distance"].default_value = 0.12
links.new(normal.outputs["Color"], bump.inputs["Height"])
links.new(bump.outputs["Normal"], bsdf.inputs["Normal"])
else:
normal_map = nodes.new("ShaderNodeNormalMap")
normal_map.inputs["Strength"].default_value = 0.52
links.new(normal.outputs["Color"], normal_map.inputs["Color"])
links.new(normal_map.outputs["Normal"], bsdf.inputs["Normal"])
return material
def link_alpha_clip(material, alpha_output, bsdf, cutoff=0.5):
"""Wire a texture's alpha into `bsdf` as a hard cut-out.
The obvious wiring — alpha straight into the Alpha socket — is wrong for
anything destined for glTF. Blender 4.2 stopped deriving a material's
alpha mode from `blend_method` (still writable, now a no-op: setting 'CLIP'
reads back 'HASHED') and made the exporter infer it from the node tree
instead. It recognises exactly a few shapes; a bare link is not one of
them, and falls through to alphaMode=BLEND. Foliage exported as BLEND
makes Cesium depth-sort thousands of leaf quads it cannot order correctly.
So build the shape the exporter looks for — `1 - (alpha < cutoff)` — which
it reads back as alphaMode=MASK with this cutoff. EEVEE gets the same
thing for free: alpha is 0 or 1 by the time it reaches the BSDF, so the
viewport shows the crisp cut-out Cesium will, not a dithered approximation.
See the exporter's `detect_alpha_clip` in
scripts/addons_core/io_scene_gltf2/blender/exp/material/search_node_tree.py.
"""
nodes = material.node_tree.nodes
links = material.node_tree.links
less_than = nodes.new("ShaderNodeMath")
less_than.operation = "LESS_THAN"
less_than.location = (-60, 320)
less_than.inputs[1].default_value = cutoff
invert = nodes.new("ShaderNodeMath")
invert.operation = "SUBTRACT"
invert.location = (110, 320)
invert.inputs[0].default_value = 1.0
links.new(alpha_output, less_than.inputs[0])
links.new(less_than.outputs["Value"], invert.inputs[1])
links.new(invert.outputs["Value"], bsdf.inputs["Alpha"])
# EEVEE Next takes its cut-out handling from surface_render_method, not
# from blend_method. 'DITHERED' still casts a leaf-shaped shadow;
# 'BLENDED' does not.
material.surface_render_method = "DITHERED"
material.alpha_threshold = cutoff
# Leaf cards are single-sided quads seen from both sides; culling
# backfaces would empty out half of every crown.
material.use_backface_culling = False
def from_spec(spec):
"""Build a material from a `catalog.MATERIALS` entry."""
if spec["kind"] == "textured":
return make_textured_material(
spec["name"], spec["diffuse"], spec["normal"],
roughness=spec.get("roughness", 0.8), scale=spec["scale"],
normal_is_bump=spec.get("normal_is_bump", False),
metallic=spec.get("metallic", 0.0),
tint=spec.get("tint"), tint_factor=spec.get("tint_factor", 0.0))
material = make_material(spec["name"], spec["color"],
spec.get("roughness", 0.8),
spec.get("metallic", 0.0))
procedural = spec.get("procedural")
if procedural:
add_procedural_surface(material, procedural["colors"],
scale=procedural["scale"],
detail=procedural["detail"],
bump_strength=procedural["bump_strength"],
object_space=procedural.get("object_space", False))
return material

126
blender/osmassets/mesh.py Normal file
View File

@@ -0,0 +1,126 @@
"""Mesh and collection construction for the Blender scene.
Requires `bpy`; only runs inside Blender.
`MeshBatch` is the workhorse: most of the scene is flat polygons and extruded
prisms, and batching them into one mesh datablock per logical group keeps the
object count (and the glTF node count) down. Callers accumulate geometry and
call `finish()` once.
"""
import math
import bpy
def new_collection(name):
collection = bpy.data.collections.new(name)
bpy.context.scene.collection.children.link(collection)
return collection
def link_object_to_collection(obj, collection):
for current in list(obj.users_collection):
current.objects.unlink(obj)
collection.objects.link(obj)
class MeshBatch:
"""Accumulates polygons/prisms into a single mesh object."""
def __init__(self, name, collection, material):
self.name = name
self.collection = collection
self.material = material
self.vertices = []
self.faces = []
def add_polygon(self, ring, z):
if len(ring) < 3:
return
if ring[0] == ring[-1]:
ring = ring[:-1]
if len(ring) < 3:
return
start = len(self.vertices)
self.vertices.extend((x, y, z) for x, y in ring)
self.faces.append(tuple(range(start, start + len(ring))))
def add_prism(self, ring, base, height):
if len(ring) < 3:
return
if ring[0] == ring[-1]:
ring = ring[:-1]
if len(ring) < 3:
return
start = len(self.vertices)
self.vertices.extend((x, y, base) for x, y in ring)
self.vertices.extend((x, y, base + height) for x, y in ring)
n = len(ring)
self.faces.append(tuple(range(start, start + n)))
self.faces.append(tuple(range(start + n, start + 2 * n)))
for i in range(n):
j = (i + 1) % n
self.faces.append((start + i, start + j, start + n + j, start + n + i))
def finish(self):
if not self.vertices:
return None
mesh = bpy.data.meshes.new(self.name + "Mesh")
mesh.from_pydata(self.vertices, [], self.faces)
mesh.materials.append(self.material)
# Foliage reads as blobby volume, so it wants smooth normals; the built
# environment wants its facets. The name prefix is the discriminator.
if self.name.startswith("Tree_") or self.name.startswith("Scrub_"):
for polygon in mesh.polygons:
polygon.use_smooth = True
mesh.update()
obj = bpy.data.objects.new(self.name, mesh)
self.collection.objects.link(obj)
return obj
def make_prism(name, ring, base, height, material, collection):
batch = MeshBatch(name, collection, material)
batch.add_prism(ring, base, height)
return batch.finish()
def add_roof(name, ring, z, material, collection):
batch = MeshBatch(name + "_Roof", collection, material)
batch.add_polygon(ring, z)
return batch.finish()
def add_wall_panel(batch, start, end, base, height, thickness=0.045, inset=0.08):
"""Add a thin inset slab along a facade edge, used for window bands."""
dx, dy = end[0] - start[0], end[1] - start[1]
length = math.hypot(dx, dy)
if length < 3.0:
return
ux, uy = dx / length, dy / length
a = (start[0] + dx * inset, start[1] + dy * inset)
b = (end[0] - dx * inset, end[1] - dy * inset)
nx, ny = -uy * thickness / 2, ux * thickness / 2
panel = [(a[0] + nx, a[1] + ny), (b[0] + nx, b[1] + ny),
(b[0] - nx, b[1] - ny), (a[0] - nx, a[1] - ny)]
batch.add_prism(panel, base, height)
def add_polyline(name, coords, projector, collection, material, width, z):
"""Bevelled curve along lon/lat coordinates; the OSM highway road fallback."""
points = [projector.xy(c) for c in coords]
if len(points) < 2:
return
curve = bpy.data.curves.new(name, "CURVE")
curve.dimensions = "3D"
curve.resolution_u = 1
curve.bevel_depth = width / 2
curve.bevel_resolution = 1
spline = curve.splines.new("POLY")
spline.points.add(len(points) - 1)
for point, (x, y) in zip(spline.points, points):
point.co = (x, y, z, 1)
obj = bpy.data.objects.new(name, curve)
collection.objects.link(obj)
obj.data.materials.append(material)

89
blender/osmassets/osm.py Normal file
View File

@@ -0,0 +1,89 @@
"""OSM XML parsing and the local metric projection.
Pure Python: no `bpy`, so this runs and tests outside Blender.
"""
import math
import xml.etree.ElementTree as ET
def tags(element):
return {t.attrib.get("k", ""): t.attrib.get("v", "")
for t in element.findall("tag")}
def parse_osm(path):
root = ET.parse(path).getroot()
bounds_node = root.find("bounds")
if bounds_node is None:
raise RuntimeError("OSM file does not contain a bounds element")
bounds = {"min_lon": float(bounds_node.attrib["minlon"]),
"min_lat": float(bounds_node.attrib["minlat"]),
"max_lon": float(bounds_node.attrib["maxlon"]),
"max_lat": float(bounds_node.attrib["maxlat"])}
nodes = {}
point_features = []
for node in root.findall("node"):
try:
node_id = int(node.attrib["id"])
coord = (float(node.attrib["lon"]), float(node.attrib["lat"]))
node_tags = tags(node)
nodes[node_id] = coord
if node_tags:
point_features.append({"id": node.attrib.get("id", ""),
"coord": coord, "tags": node_tags})
except (KeyError, ValueError):
continue
ways = []
for way in root.findall("way"):
if way.attrib.get("action") == "delete":
continue
refs = []
for ref in way.findall("nd"):
try:
refs.append(int(ref.attrib["ref"]))
except (KeyError, ValueError):
pass
coords = [nodes[r] for r in refs if r in nodes]
if len(coords) >= 2:
ways.append({"id": way.attrib.get("id", ""),
"coords": coords, "tags": tags(way)})
return bounds, ways, point_features
def parse_height(feature_tags, default):
try:
return max(0.5, float(feature_tags.get("height", default)))
except ValueError:
return default
class Projector:
"""Equirectangular projection about the centre of the OSM bounds.
Output is metres in a local ENU frame (X east, Y north), which is what both
the Blender scene and the Cesium GLB are authored in.
"""
def __init__(self, bounds):
self.bounds = bounds
self.lon0 = (bounds["min_lon"] + bounds["max_lon"]) / 2
self.lat0 = (bounds["min_lat"] + bounds["max_lat"]) / 2
self.m_per_lat = 111320.0
self.m_per_lon = 111320.0 * math.cos(math.radians(self.lat0))
def xy(self, lon_lat):
lon, lat = lon_lat
return ((lon - self.lon0) * self.m_per_lon,
(lat - self.lat0) * self.m_per_lat)
def inside(self, lon_lat, pad=0.00035):
lon, lat = lon_lat
b = self.bounds
return (b["min_lon"] - pad <= lon <= b["max_lon"] + pad and
b["min_lat"] - pad <= lat <= b["max_lat"] + pad)
def ring(self, coords):
return [self.xy(c) for c in coords]

View File

@@ -0,0 +1,13 @@
"""Scrub feature assembly (`natural=scrub`). Flat ground cover only."""
from osmassets.geom import clip_polygon
def assemble(ring, way_id, scene_xmin, scene_xmax, scene_ymin, scene_ymax,
green_c, scrub_mat, add_scrub_patch_fn):
ring = clip_polygon(ring, scene_xmin, scene_xmax, scene_ymin, scene_ymax)
focus = list(ring)
if len(ring) < 3:
return 0, focus
add_scrub_patch_fn("Scrub_" + str(way_id), ring, scrub_mat, green_c)
return 1, focus

318
blender/osmassets/tree.py Normal file
View File

@@ -0,0 +1,318 @@
"""Instanced tree assets — the model side of `--tree-style`.
Two vendored models, reduced to one runtime shape: import once, bake the source
object's orientation into a mesh copy, measure it, then link one lightweight
object per tree that reuses that datablock. Nothing is duplicated per tree, so
the .blend and the exported GLB carry each mesh and each texture exactly once
no matter how many trees are planted.
apple SpeedTree Red Delicious, 4.5k tris, alpha-cut leaf cards
fattree low-poly cartoon tree, 2.2k tris, opaque geometry
Materials are rebuilt here rather than taken from the source files, because
neither arrives usable. 57% of the apple's colour texture is transparent —
those are leaf cards, and without an alpha-clipped setup the crown renders as a
solid ball of intersecting quads. fattree ships a bare Diffuse BSDF and a
texture path that only resolves next to the original .blend.
A third style, `polyhaven`, used to live here. It appended what the Poly Haven
island_tree_01 file calls its LOD1 objects, but those are not whole trees: the
branch parts are 0.4-unit twigs and the leaf parts are flat clusters hanging
below their own origin, both meant to be scattered by the geometry-nodes setup
in that file. Planting them directly gave twigs, which is what sent us looking
for these two models. Removed along with the 78MB asset.
"""
import math
import os
from collections import namedtuple
import bpy
from osmassets.materials import link_alpha_clip
MODEL_ROOT = os.path.abspath(os.path.join(
os.path.dirname(os.path.abspath(__file__)), "..", "..",
"assets", "models",
))
APPLE_DIR = os.path.join(MODEL_ROOT, "speedtree", "apple_low")
APPLE_OBJ = os.path.join(APPLE_DIR, "RedDeliciousApple.obj")
APPLE_COLOR = os.path.join(APPLE_DIR, "textures", "apple_color_2k.png")
APPLE_NORMAL = os.path.join(APPLE_DIR, "textures", "apple_normal_2k.png")
FATTREE_DIR = os.path.join(MODEL_ROOT, "lyrog", "fattree")
FATTREE_BLEND = os.path.join(FATTREE_DIR, "fattree.blend")
FATTREE_COLOR = os.path.join(FATTREE_DIR, "textures", "fat_tree.png")
MIN_TREE_HEIGHT = 4.0
# Leaf cards are cut at half opacity: the apple atlas's alpha is near-binary
# already, so a different threshold only changes edge thickness.
ALPHA_CUTOFF = 0.5
# Golden angle. Successive trees face directions that never repeat and never
# settle into a pattern, so a tree_row reads as planted rather than stamped.
GOLDEN_TURN = 0.61803398875
SCALE_JITTER = 0.14
TILT_JITTER = math.radians(3.0)
# meshes — mesh datablocks instanced together at one transform
# height — the variant's own height, what a target height is divided by
# base_z — the variant's own ground line, what drops the trunk onto z=0
TreeVariant = namedtuple("TreeVariant", "meshes height base_z")
# --------------------------------------------------------------------------
# import plumbing
def _bake(obj, name):
"""Copy obj's mesh with its rotation and scale applied, but not its
position.
Orientation has to be baked: the OBJ importer leaves the apple's Y-up to
Z-up conversion sitting on the object, so a raw mesh copy would plant the
tree on its side. Position must *not* be, because a source file's
translation is where the artist parked the model in their own scene —
fattree sits 2.9m up in the air — and baking that in would offset every
instance by it. Dropping it costs nothing: `base_z` measures whatever
ground line the mesh ends up with, and assemble() corrects for it.
"""
mesh = obj.data.copy()
mesh.transform(obj.matrix_world.to_3x3().to_4x4())
mesh.name = name
mesh.use_fake_user = True
return mesh
def _measure(meshes):
"""Return (height, base_z) for a variant's meshes in their shared space."""
zs = [vertex.co.z for mesh in meshes for vertex in mesh.vertices]
if not zs:
return 1.0, 0.0
low, high = min(zs), max(zs)
return max(high - low, 1e-6), low
def _discard(objects):
"""Remove imported objects along with the meshes they brought in.
The _bake copies carry a fake user and survive. Dropping only the objects
would strand their original meshes at zero users, which in turn keeps the
source materials and their megabytes of texture alive in the file.
"""
for obj in objects:
mesh = obj.data if obj.type == "MESH" else None
bpy.data.objects.remove(obj, do_unlink=True)
if mesh is not None and mesh.users == 0:
bpy.data.meshes.remove(mesh)
def _purge_orphans(before_materials, before_images):
"""Drop the materials and images an import created that nothing now uses.
Both importers build a material from the source file's own description and
load its textures. We replace that material, so without this the .blend
ships a second, unreferenced copy of every 2k texture.
"""
for material in set(bpy.data.materials) - before_materials:
if material.users == 0:
bpy.data.materials.remove(material)
for image in set(bpy.data.images) - before_images:
if image.users == 0:
bpy.data.images.remove(image)
def _image(path, non_color=False):
"""Load a texture once, keyed by filename so repeat calls share it."""
key = os.path.basename(path)
image = bpy.data.images.get(key)
if image is None:
image = bpy.data.images.load(path)
image.name = key
if non_color:
image.colorspace_settings.name = "Non-Color"
return image
def _foliage_material(name, color_path, normal_path=None, alpha_clip=False,
roughness=0.72):
"""Principled setup for a textured tree, alpha-clipped when asked.
The cut-out goes through materials.link_alpha_clip rather than straight
into the Alpha socket — see that function for why the extra two nodes are
what makes the crown survive the trip to Cesium.
"""
material = bpy.data.materials.get(name)
if material:
return material
material = bpy.data.materials.new(name)
material.use_nodes = True
nodes = material.node_tree.nodes
links = material.node_tree.links
bsdf = next(n for n in nodes if n.type == "BSDF_PRINCIPLED")
bsdf.inputs["Roughness"].default_value = roughness
bsdf.inputs["Metallic"].default_value = 0.0
color_tex = nodes.new("ShaderNodeTexImage")
color_tex.image = _image(color_path)
color_tex.location = (-540, 260)
links.new(color_tex.outputs["Color"], bsdf.inputs["Base Color"])
if normal_path and os.path.exists(normal_path):
normal_tex = nodes.new("ShaderNodeTexImage")
normal_tex.image = _image(normal_path, non_color=True)
normal_tex.location = (-540, -140)
normal_map = nodes.new("ShaderNodeNormalMap")
normal_map.location = (-250, -140)
links.new(normal_tex.outputs["Color"], normal_map.inputs["Color"])
links.new(normal_map.outputs["Normal"], bsdf.inputs["Normal"])
if alpha_clip:
link_alpha_clip(material, color_tex.outputs["Alpha"], bsdf,
cutoff=ALPHA_CUTOFF)
return material
# --------------------------------------------------------------------------
# loaders — each returns [] when its model is absent, so a clean checkout
# still builds and the caller falls back to procedural trees
def _load_apple():
"""SpeedTree Red Delicious: one mesh, alpha-cut leaf cards, normal-mapped."""
if not os.path.exists(APPLE_OBJ):
return []
# Build ours first: the OBJ importer reuses an already-loaded image when the
# .mtl resolves to the same file, so the 2k textures land in the file once.
material = _foliage_material("AppleTree", APPLE_COLOR, APPLE_NORMAL,
alpha_clip=True, roughness=0.68)
before_objects = set(bpy.data.objects)
before_materials = set(bpy.data.materials)
before_images = set(bpy.data.images)
bpy.ops.wm.obj_import(filepath=APPLE_OBJ)
imported = [obj for obj in set(bpy.data.objects) - before_objects
if obj.type == "MESH"]
if not imported:
return []
meshes = []
for index, obj in enumerate(imported):
mesh = _bake(obj, "AppleTree_%02d" % index)
mesh.materials.clear()
mesh.materials.append(material)
meshes.append(mesh)
height, base_z = _measure(meshes)
_discard(imported)
_purge_orphans(before_materials, before_images)
return [TreeVariant(meshes, height, base_z)]
def _load_fattree():
"""Low-poly cartoon tree: opaque geometry, one diffuse texture.
The crown is real geometry and the texture's alpha is 1.0 everywhere, so
unlike the apple this needs no cut-out — and no normal map, which the
source does not ship.
"""
if not os.path.exists(FATTREE_BLEND):
return []
before_objects = set(bpy.data.objects)
before_materials = set(bpy.data.materials)
before_images = set(bpy.data.images)
try:
bpy.ops.wm.append(
filepath=FATTREE_BLEND + "/Object/fattree",
directory=FATTREE_BLEND + "/Object/",
files=[{"name": "fattree"}],
link=False,
)
except RuntimeError:
return []
imported = [obj for obj in set(bpy.data.objects) - before_objects
if obj.type == "MESH"]
obj = bpy.data.objects.get("fattree")
if obj is None:
_discard(imported)
_purge_orphans(before_materials, before_images)
return []
material = _foliage_material("FatTree", FATTREE_COLOR, alpha_clip=False,
roughness=0.85)
mesh = _bake(obj, "FatTree")
mesh.materials.clear()
mesh.materials.append(material)
height, base_z = _measure([mesh])
_discard(imported)
_purge_orphans(before_materials, before_images)
return [TreeVariant([mesh], height, base_z)]
LOADERS = {
"apple": _load_apple,
"fattree": _load_fattree,
}
# The styles this module can serve, for the CLI to validate against.
MODEL_STYLES = tuple(LOADERS)
def assemble(positions, collection, style="apple"):
"""Place instanced trees of `style` at `positions`.
positions is a list of (x, y, height) tuples as gathered by the two
tree-collecting loops (point nodes + tree_row samples). `height` is the
OSM height where tagged and a constant default otherwise, which means every
sample along one tree_row arrives with an identical value — the per-index
jitter below is what stops a row of forty from reading as one tree stamped
forty times.
Returns the number of trees placed, or 0 when the style's model is absent
or unknown, which is the caller's signal to fall back to procedural trees.
"""
loader = LOADERS.get(style)
if loader is None:
return 0
variants = loader()
if not variants:
return 0
for index, (x, y, target_height) in enumerate(positions):
variant = variants[index % len(variants)]
# Irrational periods stand in for an RNG: no repeat over any realistic
# tree count, and a pure function of the index, so rebuilding an area
# plants the identical forest.
scale_wobble = 1.0 + SCALE_JITTER * math.sin(index * 2.399963)
target = max(MIN_TREE_HEIGHT, target_height) * scale_wobble
factor = target / variant.height
yaw = ((index * GOLDEN_TURN) % 1.0) * math.tau
tilt_x = TILT_JITTER * math.sin(index * 1.114517)
tilt_y = TILT_JITTER * math.cos(index * 0.927295)
# Scaled and negated, the variant's own ground line drops the trunk
# onto z=0 whatever the source file used as its origin.
z = -variant.base_z * factor
for slot, mesh in enumerate(variant.meshes):
obj = bpy.data.objects.new(
"Tree_%s_%04d_%d" % (style, index, slot), mesh)
obj.location = (x, y, z)
obj.scale = (factor, factor, factor)
obj.rotation_euler = (tilt_x, tilt_y, yaw)
collection.objects.link(obj)
tris = 0
for variant in variants:
for mesh in variant.meshes:
mesh.calc_loop_triangles()
tris += len(mesh.loop_triangles)
print("Tree style %r: %d variants, %d tris per instance, %d planted"
% (style, len(variants), tris // max(1, len(variants)), len(positions)))
return len(positions)

View File

@@ -0,0 +1,15 @@
"""Water feature assembly (`natural=water` or `water=lake`)."""
from osmassets.geom import clip_polygon
from osmassets.mesh import MeshBatch
def assemble(ring, scene_xmin, scene_xmax, scene_ymin, scene_ymax,
water_c, water_mat):
ring = clip_polygon(ring, scene_xmin, scene_xmax, scene_ymin, scene_ymax)
if len(ring) < 3:
return 0
batch = MeshBatch("Lake Surface", water_c, water_mat)
batch.add_polygon(ring, 0.10)
batch.finish()
return 1

372
blender/tests/test_pure.py Normal file
View File

@@ -0,0 +1,372 @@
"""Tests for the bpy-free half of the pipeline.
python3 -m unittest discover blender/tests
These run without Blender, which is the point of the osmassets split: before
it, the only way to exercise clip_polygon or sample_tree_row was to render a
whole area and look at the picture.
The expected values are derived from the geometry, not captured from the
implementation — a test that just records current output would ratify a bug.
"""
import math
import os
import sys
import tempfile
import unittest
sys.path.insert(0, os.path.join(os.path.dirname(os.path.abspath(__file__)), ".."))
from osmassets.geom import (
clip_polygon,
distance_to_ring,
feature_in_bounds,
geometry_rings,
point_in_polygon,
polygon_area,
sample_polygon_interior,
sample_ring_boundary,
sample_tree_row,
signed_polygon_area,
)
from osmassets.osm import Projector, parse_height, parse_osm, tags
SQUARE = [(0.0, 0.0), (10.0, 0.0), (10.0, 10.0), (0.0, 10.0)]
class GeometryRingsTest(unittest.TestCase):
def test_polygon_keeps_only_the_exterior_ring(self):
geometry = {"type": "Polygon", "coordinates": [["outer"], ["hole"]]}
self.assertEqual(geometry_rings(geometry), [["outer"]])
def test_multipolygon_takes_each_exterior_ring(self):
geometry = {"type": "MultiPolygon",
"coordinates": [[["a"], ["a hole"]], [["b"]]]}
self.assertEqual(geometry_rings(geometry), [["a"], ["b"]])
def test_unsupported_and_empty_geometry(self):
self.assertEqual(geometry_rings(None), [])
self.assertEqual(geometry_rings({}), [])
self.assertEqual(geometry_rings({"type": "LineString",
"coordinates": [[0, 0], [1, 1]]}), [])
self.assertEqual(geometry_rings({"type": "MultiPolygon",
"coordinates": [[], [["b"]]]}), [["b"]])
class ClipPolygonTest(unittest.TestCase):
def test_polygon_inside_the_box_is_unchanged(self):
clipped = clip_polygon(SQUARE, -1.0, 11.0, -1.0, 11.0)
self.assertEqual([(round(x, 6), round(y, 6)) for x, y in clipped], SQUARE)
def test_half_outside_polygon_is_cut_at_the_boundary(self):
clipped = clip_polygon(SQUARE, 0.0, 5.0, 0.0, 10.0)
self.assertTrue(all(x <= 5.0 + 1e-9 for x, _ in clipped))
# A 10x10 square clipped to half its width is a 5x10 rectangle.
self.assertAlmostEqual(polygon_area(clipped), 50.0, places=6)
def test_polygon_fully_outside_collapses(self):
self.assertEqual(clip_polygon(SQUARE, 20.0, 30.0, 20.0, 30.0), [])
def test_degenerate_input(self):
self.assertEqual(clip_polygon([], 0, 1, 0, 1), [])
self.assertEqual(clip_polygon([(0.0, 0.0), (1.0, 1.0)], 0, 1, 0, 1), [])
def test_axis_aligned_edge_does_not_divide_by_zero(self):
# A vertical edge crossing the x clip plane exercises the b[0] == a[0]
# guard in the intersection lambdas.
ring = [(5.0, -5.0), (5.0, 5.0), (-5.0, 5.0), (-5.0, -5.0)]
clipped = clip_polygon(ring, 0.0, 10.0, 0.0, 10.0)
self.assertAlmostEqual(polygon_area(clipped), 25.0, places=6)
class PolygonAreaTest(unittest.TestCase):
def test_square(self):
self.assertAlmostEqual(polygon_area(SQUARE), 100.0)
def test_winding_does_not_change_the_sign(self):
self.assertAlmostEqual(polygon_area(list(reversed(SQUARE))), 100.0)
self.assertGreater(signed_polygon_area(SQUARE), 0.0)
self.assertLess(signed_polygon_area(list(reversed(SQUARE))), 0.0)
def test_degenerate(self):
self.assertEqual(polygon_area([(0.0, 0.0), (1.0, 1.0)]), 0.0)
class SampleRingBoundaryTest(unittest.TestCase):
def test_samples_closed_boundary_evenly(self):
samples = sample_ring_boundary(SQUARE, spacing=10.0)
self.assertEqual(len(samples), 4)
self.assertEqual([(round(x, 6), round(y, 6)) for x, y, _, _ in samples],
[(5.0, 0.0), (10.0, 5.0), (5.0, 10.0), (0.0, 5.0)])
def test_repeated_closing_point_is_ignored(self):
open_samples = sample_ring_boundary(SQUARE, spacing=10.0)
closed_samples = sample_ring_boundary(SQUARE + [SQUARE[0]], spacing=10.0)
self.assertEqual(open_samples, closed_samples)
def test_inset_moves_samples_inside_for_either_winding(self):
ccw = sample_ring_boundary(SQUARE, spacing=10.0, inset=1.0)
cw = sample_ring_boundary(list(reversed(SQUARE)), spacing=10.0, inset=1.0)
self.assertTrue(all(point_in_polygon((x, y), SQUARE) for x, y, _, _ in ccw))
self.assertTrue(all(point_in_polygon((x, y), SQUARE) for x, y, _, _ in cw))
self.assertEqual([(round(x, 6), round(y, 6)) for x, y, _, _ in ccw],
[(5.0, 1.0), (9.0, 5.0), (5.0, 9.0), (1.0, 5.0)])
def test_max_samples_reduces_density(self):
samples = sample_ring_boundary(SQUARE, spacing=1.0, max_samples=5)
self.assertEqual(len(samples), 5)
def test_degenerate_input(self):
self.assertEqual(sample_ring_boundary([(0.0, 0.0)], spacing=1.0), [])
self.assertEqual(sample_ring_boundary(SQUARE, spacing=0.0), [])
class SamplePolygonInteriorTest(unittest.TestCase):
def test_samples_are_inside_and_clear_of_edges(self):
samples = sample_polygon_interior(SQUARE, spacing=3.0, edge_clearance=1.0,
seed=42)
self.assertTrue(samples)
for x, y, _ in samples:
self.assertTrue(point_in_polygon((x, y), SQUARE))
self.assertGreaterEqual(distance_to_ring((x, y), SQUARE), 1.0)
def test_seed_is_deterministic(self):
first = sample_polygon_interior(SQUARE, spacing=3.0, seed=7)
second = sample_polygon_interior(SQUARE, spacing=3.0, seed=7)
self.assertEqual(first, second)
def test_max_samples_reduces_density(self):
samples = sample_polygon_interior(SQUARE, spacing=1.0, max_samples=4,
seed=99)
self.assertEqual(len(samples), 4)
def test_degenerate_input(self):
self.assertEqual(sample_polygon_interior([(0.0, 0.0)], spacing=1.0), [])
self.assertEqual(sample_polygon_interior(SQUARE, spacing=0.0), [])
class PointInPolygonTest(unittest.TestCase):
def test_inside_and_outside(self):
self.assertTrue(point_in_polygon((5.0, 5.0), SQUARE))
self.assertFalse(point_in_polygon((15.0, 5.0), SQUARE))
self.assertFalse(point_in_polygon((5.0, -0.5), SQUARE))
def test_concave_notch_is_excluded(self):
# An L shape: the notch at (8, 8) is outside even though it sits inside
# the bounding box.
shape = [(0.0, 0.0), (10.0, 0.0), (10.0, 5.0),
(5.0, 5.0), (5.0, 10.0), (0.0, 10.0)]
self.assertTrue(point_in_polygon((2.0, 8.0), shape))
self.assertFalse(point_in_polygon((8.0, 8.0), shape))
class DistanceToRingTest(unittest.TestCase):
def test_distance_is_to_the_edge_not_the_interior(self):
# Centre of the square: 5m from every edge, even though it is inside.
self.assertAlmostEqual(distance_to_ring((5.0, 5.0), SQUARE), 5.0)
self.assertAlmostEqual(distance_to_ring((1.0, 5.0), SQUARE), 1.0)
def test_outside_point(self):
self.assertAlmostEqual(distance_to_ring((-3.0, 5.0), SQUARE), 3.0)
def test_closes_the_ring(self):
# Nearest edge is the implicit closing segment from (0,10) back to (0,0).
self.assertAlmostEqual(distance_to_ring((-2.0, 9.0), SQUARE), 2.0)
def test_repeated_vertex_does_not_divide_by_zero(self):
ring = [(0.0, 0.0), (0.0, 0.0), (4.0, 0.0)]
self.assertAlmostEqual(distance_to_ring((2.0, 3.0), ring), 3.0)
class SampleTreeRowTest(unittest.TestCase):
def test_even_spacing_along_a_straight_line(self):
samples = sample_tree_row([(0.0, 0.0), (10.0, 0.0)], spacing=5.0, height=6.0)
self.assertEqual([(round(x, 6), round(y, 6)) for x, y, _ in samples],
[(0.0, 0.0), (5.0, 0.0), (10.0, 0.0)])
self.assertTrue(all(h == 6.0 for _, _, h in samples))
def test_spacing_carries_across_segment_joins(self):
# Two 3m segments with 4m spacing: the second sample must land 1m into
# the second segment, not restart at its origin.
samples = sample_tree_row([(0.0, 0.0), (3.0, 0.0), (6.0, 0.0)],
spacing=4.0, height=5.0)
xs = [round(x, 6) for x, _, _ in samples]
self.assertEqual(xs, [0.0, 4.0, 6.0])
def test_trailing_point_is_skipped_when_it_would_double_plant(self):
# Endpoint sits 0.2m past the last sample, well under spacing * 0.45.
samples = sample_tree_row([(0.0, 0.0), (5.2, 0.0)], spacing=5.0, height=5.0)
self.assertEqual([round(x, 6) for x, _, _ in samples], [0.0, 5.0])
def test_zero_length_segment_is_skipped(self):
samples = sample_tree_row([(0.0, 0.0), (0.0, 0.0), (10.0, 0.0)],
spacing=5.0, height=5.0)
self.assertEqual([round(x, 6) for x, _, _ in samples], [0.0, 5.0, 10.0])
def test_too_few_points(self):
self.assertEqual(sample_tree_row([(0.0, 0.0)], spacing=5.0, height=5.0), [])
BOUNDS = {"min_lon": 114.0, "min_lat": 30.0, "max_lon": 114.01, "max_lat": 30.01}
class ProjectorTest(unittest.TestCase):
def setUp(self):
self.projector = Projector(BOUNDS)
def test_centre_of_bounds_is_the_origin(self):
x, y = self.projector.xy((114.005, 30.005))
self.assertAlmostEqual(x, 0.0, places=6)
self.assertAlmostEqual(y, 0.0, places=6)
def test_axes_point_east_and_north(self):
east, _ = self.projector.xy((114.006, 30.005))
_, north = self.projector.xy((114.005, 30.006))
self.assertGreater(east, 0.0)
self.assertGreater(north, 0.0)
def test_longitude_metres_shrink_with_latitude(self):
self.assertAlmostEqual(
self.projector.m_per_lon,
111320.0 * math.cos(math.radians(30.005)),
places=6,
)
self.assertLess(self.projector.m_per_lon, self.projector.m_per_lat)
def test_inside_honours_the_pad(self):
self.assertTrue(self.projector.inside((114.005, 30.005)))
# Default pad is 0.00035 degrees, so just outside the box still counts.
self.assertTrue(self.projector.inside((114.0102, 30.005)))
self.assertFalse(self.projector.inside((114.02, 30.005)))
self.assertFalse(self.projector.inside((114.0102, 30.005), pad=0.0))
def test_ring_projects_every_coordinate(self):
ring = self.projector.ring([(114.0, 30.0), (114.01, 30.01)])
self.assertEqual(len(ring), 2)
self.assertLess(ring[0][0], 0.0)
self.assertGreater(ring[1][0], 0.0)
class FeatureInBoundsTest(unittest.TestCase):
def setUp(self):
self.projector = Projector(BOUNDS)
def test_polygon_with_one_inside_vertex_counts(self):
feature = {"geometry": {"type": "Polygon", "coordinates": [[
[120.0, 40.0], [114.005, 30.005], [120.0, 40.0]]]}}
self.assertTrue(feature_in_bounds(feature, self.projector))
def test_feature_fully_outside(self):
feature = {"geometry": {"type": "Polygon", "coordinates": [[
[120.0, 40.0], [120.1, 40.1], [120.0, 40.0]]]}}
self.assertFalse(feature_in_bounds(feature, self.projector))
def test_missing_geometry(self):
self.assertFalse(feature_in_bounds({}, self.projector))
class ParseHeightTest(unittest.TestCase):
def test_reads_the_tag(self):
self.assertEqual(parse_height({"height": "24"}, 12.0), 24.0)
def test_missing_tag_falls_back(self):
self.assertEqual(parse_height({}, 12.0), 12.0)
def test_unparsable_tag_falls_back(self):
self.assertEqual(parse_height({"height": "about 20m"}, 12.0), 12.0)
def test_clamped_to_half_a_metre(self):
self.assertEqual(parse_height({"height": "0.1"}, 12.0), 0.5)
self.assertEqual(parse_height({"height": "-5"}, 12.0), 0.5)
OSM_SAMPLE = """<?xml version='1.0' encoding='UTF-8'?>
<osm version='0.6'>
<bounds minlon='114.0' minlat='30.0' maxlon='114.01' maxlat='30.01'/>
<node id='1' lon='114.001' lat='30.001'/>
<node id='2' lon='114.002' lat='30.002'/>
<node id='3' lon='114.003' lat='30.003'/>
<node id='4' lon='114.004' lat='30.004'>
<tag k='natural' v='tree'/>
<tag k='height' v='7'/>
</node>
<node id='bad' lon='oops' lat='30.0'/>
<way id='10'>
<nd ref='1'/><nd ref='2'/><nd ref='3'/>
<tag k='building' v='yes'/>
</way>
<way id='11' action='delete'>
<nd ref='1'/><nd ref='2'/>
<tag k='building' v='yes'/>
</way>
<way id='12'>
<nd ref='1'/><nd ref='999'/>
</way>
</osm>
"""
class ParseOsmTest(unittest.TestCase):
def setUp(self):
handle = tempfile.NamedTemporaryFile("w", suffix=".osm", delete=False,
encoding="utf-8")
handle.write(OSM_SAMPLE)
handle.close()
self.path = handle.name
def tearDown(self):
os.unlink(self.path)
def test_bounds(self):
bounds, _, _ = parse_osm(self.path)
self.assertEqual(bounds, {"min_lon": 114.0, "min_lat": 30.0,
"max_lon": 114.01, "max_lat": 30.01})
def test_only_tagged_nodes_become_point_features(self):
_, _, points = parse_osm(self.path)
self.assertEqual([p["id"] for p in points], ["4"])
self.assertEqual(points[0]["tags"], {"natural": "tree", "height": "7"})
def test_deleted_ways_are_dropped(self):
_, ways, _ = parse_osm(self.path)
self.assertNotIn("11", [w["id"] for w in ways])
def test_way_below_two_resolvable_nodes_is_dropped(self):
# Way 12 references a node that does not exist, leaving one coordinate.
_, ways, _ = parse_osm(self.path)
self.assertEqual([w["id"] for w in ways], ["10"])
self.assertEqual(len(ways[0]["coords"]), 3)
self.assertEqual(ways[0]["tags"], {"building": "yes"})
def test_unparsable_node_is_skipped_not_fatal(self):
_, _, points = parse_osm(self.path)
self.assertNotIn("bad", [p["id"] for p in points])
def test_missing_bounds_is_an_error(self):
handle = tempfile.NamedTemporaryFile("w", suffix=".osm", delete=False,
encoding="utf-8")
handle.write("<osm version='0.6'></osm>")
handle.close()
try:
with self.assertRaises(RuntimeError):
parse_osm(handle.name)
finally:
os.unlink(handle.name)
class TagsTest(unittest.TestCase):
def test_reads_key_value_children(self):
import xml.etree.ElementTree as ET
element = ET.fromstring(
"<way><tag k='building' v='yes'/><tag k='height' v='9'/></way>")
self.assertEqual(tags(element), {"building": "yes", "height": "9"})
def test_untagged_element(self):
import xml.etree.ElementTree as ET
self.assertEqual(tags(ET.fromstring("<way/>")), {})
if __name__ == "__main__":
unittest.main()

View File

@@ -0,0 +1,171 @@
"""Dump a stable structural digest of a .blend built by generate_scene.py.
Run inside Blender:
Blender --background --factory-startup \
--python blender/tools/scene_digest.py -- \
--blend /path/to/scene.blend --out /path/to/digest.json
The digest is the parity contract for the osmassets refactor: it must stay
byte-identical across a pure restructuring. Fields that a control run (same
code, run twice) proves unstable belong in UNSTABLE_* below rather than in the
digest, otherwise the check is noise and gets ignored.
Floats are rounded to 6 decimals: Blender round-trips them through single
precision, so the last digits of a repr are not a meaningful signal.
"""
import json
import os
import sys
import bpy
# Object-level custom properties Blender adds on its own; not ours to compare.
IGNORED_PROP_KEYS = {"_RNA_UI", "cycles"}
def cli_args():
values = {"blend": None, "out": None}
argv = sys.argv[sys.argv.index("--") + 1:] if "--" in sys.argv else []
i = 0
while i < len(argv):
if argv[i].startswith("--") and i + 1 < len(argv):
values[argv[i][2:]] = argv[i + 1]
i += 2
else:
i += 1
for key in ("blend", "out"):
if not values.get(key):
raise RuntimeError("--%s is required" % key)
return values
def rounded(value):
"""Normalise Blender's float/vector/array soup into plain JSON."""
if isinstance(value, float):
return round(value, 6)
if isinstance(value, (int, str, bool)) or value is None:
return value
if hasattr(value, "__len__") and not isinstance(value, (str, bytes)):
return [rounded(item) for item in value]
return str(value)
def custom_props(datablock):
out = {}
for key in sorted(datablock.keys()):
if key in IGNORED_PROP_KEYS:
continue
try:
out[key] = rounded(datablock[key])
except (TypeError, ValueError):
out[key] = "<unreadable>"
return out
def material_digest(material):
node_types = {}
if material.use_nodes and material.node_tree:
for node in material.node_tree.nodes:
node_types[node.type] = node_types.get(node.type, 0) + 1
entry = {
"name": material.name,
"diffuse_color": rounded(material.diffuse_color),
"use_nodes": material.use_nodes,
# Node identity is unstable (Blender names them Mix.001, Mix.002 …
# depending on creation order across datablocks), so compare the
# type histogram and the link count instead of the graph itself.
"node_types": dict(sorted(node_types.items())),
"link_count": (len(material.node_tree.links)
if material.use_nodes and material.node_tree else 0),
"props": custom_props(material),
}
if material.use_nodes and material.node_tree:
for node in material.node_tree.nodes:
if node.type != "BSDF_PRINCIPLED":
continue
for socket in ("Base Color", "Roughness", "Metallic"):
if socket in node.inputs:
entry["bsdf_" + socket.replace(" ", "_").lower()] = rounded(
node.inputs[socket].default_value)
break
return entry
def object_digest(obj):
entry = {
"name": obj.name,
"type": obj.type,
"collections": sorted(c.name for c in obj.users_collection),
"location": rounded(obj.location),
"rotation_euler": rounded(obj.rotation_euler),
"scale": rounded(obj.scale),
"data": obj.data.name if obj.data else None,
"materials": [slot.material.name if slot.material else None
for slot in obj.material_slots],
"modifiers": [(m.name, m.type) for m in obj.modifiers],
"props": custom_props(obj),
}
if obj.type == "MESH":
mesh = obj.data
entry["vertices"] = len(mesh.vertices)
entry["polygons"] = len(mesh.polygons)
entry["uv_layers"] = [layer.name for layer in mesh.uv_layers]
entry["smooth_polygons"] = sum(1 for p in mesh.polygons if p.use_smooth)
# Bounding box catches geometry that moved without changing topology;
# a vertex-by-vertex hash would be exact but too brittle to act on.
entry["bound_box"] = [rounded(corner) for corner in obj.bound_box]
elif obj.type == "CURVE":
entry["splines"] = len(obj.data.splines)
entry["points"] = sum(len(s.points) for s in obj.data.splines)
entry["bevel_depth"] = rounded(obj.data.bevel_depth)
elif obj.type == "LIGHT":
entry["light_type"] = obj.data.type
entry["energy"] = rounded(obj.data.energy)
elif obj.type == "CAMERA":
entry["lens"] = rounded(obj.data.lens)
entry["clip"] = [rounded(obj.data.clip_start), rounded(obj.data.clip_end)]
return entry
def digest(blend_path):
bpy.ops.wm.open_mainfile(filepath=blend_path)
scene = bpy.context.scene
return {
"scene": {
"name": scene.name,
"engine": scene.render.engine,
"resolution": [scene.render.resolution_x, scene.render.resolution_y],
"world_color": rounded(scene.world.color) if scene.world else None,
"camera": scene.camera.name if scene.camera else None,
"props": custom_props(scene),
},
"collections": sorted(c.name for c in bpy.data.collections),
"counts": {
"objects": len(bpy.data.objects),
"meshes": len(bpy.data.meshes),
"materials": len(bpy.data.materials),
"images": len(bpy.data.images),
},
"objects": [object_digest(obj)
for obj in sorted(bpy.data.objects, key=lambda o: o.name)],
"materials": [material_digest(mat)
for mat in sorted(bpy.data.materials, key=lambda m: m.name)],
"images": sorted(image.name for image in bpy.data.images),
}
if __name__ == "__main__":
args = cli_args()
result = digest(args["blend"])
os.makedirs(os.path.dirname(os.path.abspath(args["out"])), exist_ok=True)
with open(args["out"], "w", encoding="utf-8") as handle:
json.dump(result, handle, ensure_ascii=False, indent=2, sort_keys=True)
handle.write("\n")
print("DIGEST_DONE", json.dumps({
"blend": args["blend"], "out": args["out"],
"objects": len(result["objects"]),
"materials": len(result["materials"]),
}))

View File

@@ -0,0 +1,32 @@
{
"id": "hanyang-block",
"input": "/Users/que01/Desktop/汉阳区区块.osm",
"outputRoot": "/Users/que01/osm2streets-qgis-workflow/outputs",
"qgisApp": "/Applications/QGIS.app",
"blenderApp": "/Applications/Blender.app",
"stages": {
"intermediates": true,
"blender": true,
"cesium": true
},
"qgis": {
"arrowScale": 0.8,
"clipPad": 0.002,
"canvasPad": 0.001,
"previewPad": 0.0007,
"canvasExtent": null,
"previewExtent": null,
"layerPrefix": "osm2streets"
},
"osm2streets": {
"debug_each_step": false,
"dual_carriageway_experiment": false,
"sidepath_zipping_experiment": false,
"inferred_sidewalks": true,
"osm2lanes": true
},
"blender": {
"treeStyle": "natural",
"officeOverrides": ""
}
}

View File

@@ -0,0 +1,35 @@
{
"id": "nantaizi-lake-innovation-valley",
"input": "/Users/que01/Desktop/南台子湖创新谷OSM.osm",
"outputRoot": "/Users/que01/osm2streets-qgis-workflow/outputs",
"qgisApp": "/Applications/QGIS.app",
"blenderApp": "/Applications/Blender.app",
"stages": {
"intermediates": true,
"blender": true,
"cesium": true
},
"qgis": {
"arrowScale": 0.8,
"arrowMergeTriangles": true,
"arrowOutlineSimplifyMeters": 0.05,
"intersectionCornerSourceMaxDimensionMeters": 2.6,
"clipPad": 0.002,
"canvasPad": 0.001,
"previewPad": 0.0007,
"canvasExtent": null,
"previewExtent": null,
"layerPrefix": "osm2streets"
},
"osm2streets": {
"debug_each_step": false,
"dual_carriageway_experiment": false,
"sidepath_zipping_experiment": false,
"inferred_sidewalks": true,
"osm2lanes": true
},
"blender": {
"treeStyle": "apple",
"officeOverrides": ""
}
}

View File

@@ -1,17 +1,26 @@
{
"qgisApp": "/Applications/QGIS.app",
"id": "my-area",
"input": "/absolute/path/to/input.osm",
"outDir": "/absolute/path/to/osm2streets_web_out",
"gpkg": "/absolute/path/to/output.gpkg",
"project": "/absolute/path/to/output.qgz",
"preview": "/absolute/path/to/output_preview.png",
"arrowScale": 0.8,
"clipPad": 0.002,
"canvasPad": 0.001,
"previewPad": 0.0007,
"canvasExtent": null,
"previewExtent": null,
"layerPrefix": "osm2streets",
"outputRoot": "/absolute/path/to/outputs",
"qgisApp": "/Applications/QGIS.app",
"blenderApp": "/Applications/Blender.app",
"stages": {
"intermediates": true,
"blender": true,
"cesium": true
},
"qgis": {
"arrowScale": 0.8,
"arrowMergeTriangles": true,
"arrowOutlineSimplifyMeters": 0.05,
"intersectionCornerSourceMaxDimensionMeters": 2.6,
"clipPad": 0.002,
"canvasPad": 0.001,
"previewPad": 0.0007,
"canvasExtent": null,
"previewExtent": null,
"layerPrefix": "osm2streets"
},
"osm2streets": {
"debug_each_step": false,
"dual_carriageway_experiment": false,
@@ -20,10 +29,7 @@
"osm2lanes": true
},
"blender": {
"scene": "/absolute/path/to/scene.blend",
"render": "/absolute/path/to/preview.png",
"glb": "/absolute/path/to/cesium.glb",
"metadata": "/absolute/path/to/cesium.json",
"office_overrides": ""
"treeStyle": "natural",
"officeOverrides": ""
}
}
}

View File

@@ -1,5 +1,102 @@
# Changelog
## 2026-07-31远看发黑的真正原因反照率没被提亮
(二)里加的 emissive 提到 0.8 仍然发黑。原因是 emissive 乘的是本来就很暗的
反照率0.8 × 深绿 ≈ sRGB `[0.22, 0.31, 0.11]`,还是暗的。**要提的是反照率本身。**
用户给的 Cesium 截图定位了问题:草是亮黄绿、建筑近白、路面浅灰,只有树是暗的。
这棵树的图集叶片本来就是深绿——绿色系像素均值 sRGB `[0.249, 0.35, 0.12]`。按真实
反照率渲染是对的,但**场景里其他材质都被手工提亮过**`EXPORT_TINTS` 草 0.72、
带肋墙面 0.86`EXPORT_EMISSION_OVERRIDES` 建筑 0.18),全是针对 Cesium 偏白的
默认光照调出来的。新资产没调过,是唯一一个如实渲染的东西,放在旁边就显得发黑。
- 新增 `FOLIAGE_ALBEDO_GAIN = 2.1`,在抠图植被的 dilate 那一遍里顺带乘上去。
用增益而不是 tint其他材质是单一表面所以 tint 合适,而这是一张同时装着叶片、
树皮、果实的图集,往绿色混会把树干也染绿。缩放保留色相关系,只是把整体抬到和
邻居一样的曝光。叶片 sRGB `[0.249, 0.35, 0.12]``[0.36, 0.50, 0.18]`
过曝到纯白的像素只占 0.4%
- `FOLIAGE_EMISSION` 回调到 0.25:它的职责只是给背光面兜底,不是主要提亮手段
- 增益之后远看仍偏灰绿,再加 `FOLIAGE_SATURATION = 1.75`,绕各像素自身
Rec.709 亮度做饱和度拉伸。这张图集本来就偏灰(平均饱和度 0.22),而远看时
mip 会把叶片、树皮和缝隙混在一起,越小越往中性靠。绕亮度拉伸能把叶片推绿而
基本不动本来就中性的部分,也没有绿色 tint 强加给树干的色相偏移——树皮只是变
暖一点。叶片 sRGB → `[0.262, 0.529, 0.021]`,整体饱和度 0.22 → 0.34
- 新增 `triangulate_mesh()``export_tangents` 打开后刷了 55 行
「切向空间只能只算三角/四边形」——`MeshBatch` 建的 OSM 轮廓都是 n-gon
Blender 只能给三角/四边形算切线。glTF 本来就只有三角形,导出时无论如何都会
三角化,所以提前做不改变任何一个输出三角形(实测三角数 51719 前后一致),
但切线从 49/102 变成 102/102警告归零
排查中被数据排除的假设,记下来免得重走:贴图颜色全链路逐位一致(不是 gamma
模拟 mip 链可见像素亮度 0.127→0.124(不是 mipmap法线贴图抠图区是干净平面法线
(不是法线污染);把导出的 GLB 重新导入 Blender 渲染,树是正常的(文件没问题)。
## 2026-07-31远看整棵树发黑
黑色色块修掉后Cesium 里近看正常、远看整棵树是暗色块。逐项排查:
- 贴图颜色全链路无偏移:源 4k / 降采样 2k / GLB 里导出的 PNGopaque 均值都是 sRGB `[0.409, 0.406, 0.362]`,不是 gamma 问题
- 模拟 GPU mip 链(逐级 box 降采样,按 0.5 cutoff 取可见像素):可见像素亮度 mip0→mip6 只从 0.127 变到 0.124,覆盖率稳定在 0.42,不是 mipmap 变暗
- 法线贴图抠图区域是干净的平面法线 `[0.494, 0.512, 0.988]`无黑像素mip 后趋于更平,不是法线污染
真实原因是**缺少 emissive 补偿**。预览页没有配置任何环境贴图(`skyBox` / `skyAtmosphere` / `sun` 全部关闭Cesium 只剩一个很弱的默认球谐环境光,所以太阳直射不到的面接近全黑——这正是 `EXPORT_EMISSION_OVERRIDES` 存在的原因,建筑 0.18、程序化树冠 0.015~0.02。树冠绝大部分是背光的叶片卡片远看整体塌成一团暗色而近看能看到向阳面所以还行。apple 材质不在那张表里,一点补偿都没有。
- 抠图植被改为把 diffuse 接回 Emission Color强度 `FOLIAGE_EMISSION = 0.22`。用带贴图的自发光而不是平坦颜色:常量会把树干也染成叶子绿,而这样每个像素的下限是它自身反照率的一个比例。不增加字节,导出器让 `emissiveTexture` 指向 base color 已经在用的那张图
- glTF 导出打开 `export_tangents`apple 材质有法线贴图但图元没有 TANGENT缺失时由渲染器自行推导切线而在双面薄片叶子卡上这个推导不可靠。全场景 49/102 个图元带上切线顺带修正建筑法线贴图GLB +0.83MB
如果远处仍偏暗,调 `export_cesium.py``FOLIAGE_EMISSION` 一个常量即可。
## 2026-07-31
- 树渲染改用两个第三方模型,`tree_style` 新增 `apple` / `fattree`
- `apple` — SpeedTree Red Delicious4475 tris叶片 alpha 抠图 + 法线贴图4k 贴图降采样为 2k
- `fattree` — 低面数卡通树2238 tris实体几何
- 高度不再用魔数缩放,改为按模型自身包围盒归一化到目标高度,树根落在 z=0
- 每棵树按序号做确定性抖动:缩放 ±14%、黄金角偏航、±3° 倾斜,成排的行道树不再是同一棵树盖章
- 移除 `polyhaven` 树样式和 island_tree_01 资产76MB+ `blender/tools/ingest_tree.py`。该样式 append 的 `*_LOD1` 对象并不是完整的树:枝干只有 0.41 单位高,叶片是挂在原点下方的平面簇,原本是给源文件里的几何节点散布用的,直接种下去只有树枝
- 修复 Cesium 导出丢失 alpha 抠图:`make_export_material` 原本把 Alpha 恒定写死为 1.0,叶片卡片整块导出,而 SpeedTree 图集抠掉的区域是纯黑,在 Cesium 里表现为黑色色块
- Blender 4.2 起 glTF 导出不再读 `blend_method`(仍可写但已失效,写 `CLIP` 读回来是 `HASHED`),改为从节点树推断 alpha 模式。新增 `materials.link_alpha_clip()` 构造导出器识别的 `1 - (alpha < cutoff)` 节点形状,得到 `alphaMode=MASK`,同时 EEVEE 里也变成硬边抠图
- 新增 `alpha_dilated_image()`:把不透明像素的颜色向抠图区域外扩 8 圈。97.5% 的透明像素是纯黑Cesium 生成 mipmap 时会把黑色平均进叶片边缘。叶片边缘相邻的纯黑像素占比从 10.3% 降到 0.6%
- 修复导出器对实例化网格重复包装材质:材质槽在 mesh 上181 棵树共享一个 datablock第一棵替换后其余 180 棵会把结果再包一次,产生 `Cesium Cesium Cesium ...` 的材质名且烘焙图缓存按材质名索引每轮都会再嵌一份同样的贴图。GLB 22.46MB → 20.76MBmaterials 270 → 23
- `source_alpha_clipped()` 同时要求「材质连了 Alpha」和「贴图确实有抠图」shrub_02 从 glTF 带进来一个 Math 节点接在 Alpha 上,但它的 JPEG 贴图 alpha 全是 1.0,只判断前者会把它误判为抠图材质,白白重编码成 1.2MB PNG 并让 Cesium 对 270 丛草做 alpha test
- 场景新增 `tree_style` / `tree_style_used` 属性;模型资产缺失时自动回落到 `natural`
## 2026-07-30
- 重构 Blender 脚本为 `osmassets` 包:`generate_scene.py` 从 1271 行缩减到 826 行 (-35%),纯函数可脱离 Blender 测试 (42 个 unittest)
- `osm.py` — OSM 解析 / Projector / parse_height无 bpy
- `geom.py` — 平面几何纯函数(无 bpy
- `materials.py` — Blender 材质创建
- `mesh.py` — MeshBatch / prism / roof / polyline
- `catalog.py` — 道路图层和材质规格的唯一定义源,含 cesium 导出参数
- 要素注册表:`water.py` / `grass.py` / `scrub.py` / `tree.py`,各导出一个 `assemble()` 函数
- 新增 parity 工具链:`scene_digest.py` + `glb-digest.js` + `parity.js`,两区域全 PARITY OK
- 接入 Poly Haven island_tree_01 真实扫描树模型:`tree_style` 新增 `polyhaven` 选项(已于 2026-07-31 移除,见上)
- 新增 `blender/tools/ingest_tree.py` 用于离线减面导出树模型(已于 2026-07-31 移除)
- `nantaizi-lake-innovation-valley` 配置默认切换为 `treeStyle: polyhaven`(现为 `apple`
## 2026-07-28
- 新增 `reimport` 阶段(`scripts/reimport-gpkg.js`),把 QGIS 手工修正过的 `<area-id>.gpkg` 回导为 `osm2streets_web_out/*.geojson` 并重建 `osm2streets_scene.geojson` / `osm2streets_scene_style.json`。手工修正流程从「`ogr2ogr` 循环 + 内联 node 脚本 + `npm run build`」三步压缩为一条命令:
```bash
npm run build -- --config config/areas/<area-id>.json --stages reimport,blender,cesium
```
- `reimport` 先把全部图层导出到临时目录并逐个校验,全部通过才写回输出目录:`ogr2ogr` 对不存在的图层退出码非 0 但仍会留下 0 字节文件,逐图层直接覆盖会静默损坏数据。
- `intermediates` 与 `reimport` 同时指定时直接报错:前者会用 OSM 重建 GeoPackage正好抹掉后者要读回的手工修改。`reimport` 不含在 `all` 中。
- 新增 `scripts/lib/scene-layers.js` 作为 9 个渲染图层的唯一定义源id、`z_index`、配色、描边宽度)。此前该表在合并场景、场景样式 JSON、生成的 QGIS 工程、README 手工流程中各有一份副本,改一处漏其余会导致图层叠放顺序错误并流入 Blender/Cesium。构建产物GeoJSON、GeoPackage、`.qgz` 符号、样式 JSON与改动前逐字节一致。
## 2026-07-27
- 实验分支新增 Cesium 车辆巡航预览:从 OSM 可行驶 `highway` 提取 bounds 内路线,输出 `<area-id>-vehicle-route.json`,并在预览页中驱动车辆循环移动。
- 巡航路线从道路中心线向右偏移约 1.3 米,车辆模型改为无 logo 的轻量预览车。
- 车辆巡航预览支持多辆车同时行驶,并通过 `Vehicle` 下拉框选择 Follow 目标。
- 将项目主入口重构为区域资产管线:`scripts/build-area.js`。
- 新增 `config/areas/nantaizi-lake-innovation-valley.json` 和 `config/areas/hanyang-block.json`,支持按 OSM 输入生成独立输出目录。
- `npm run build` 现在默认走区域资产管线;旧 QGIS 管线保留为 `npm run build:qgis`。
- `cesium` 阶段会生成 `<area-id>-cesium-preview.html` 本地预览页。
- 修复 Blender 脚本对 `--tree-style` 和 `--office-overrides` 这类连字符参数的解析。
## 2026-07-24
- Blender 场景生成器通用化:`generate_nantaizi.py` → `generate_scene.py`
@@ -20,4 +117,4 @@
- Added [config/hanyang-block.json](/Users/que01/osm2streets-qgis-workflow/config/hanyang-block.json) for `/Users/que01/Desktop/汉阳区区块.osm`.
- Added [config/examples/template.json](/Users/que01/osm2streets-qgis-workflow/config/examples/template.json) for new areas.
- Updated OSM node coordinate parsing to support both single-quoted and double-quoted XML attributes.
- Verified QGIS outputs for both a smaller Overpass-style XML input and a larger JOSM-generated `.osm` input.
- Verified QGIS outputs for both a smaller Overpass-style XML input and a larger JOSM-generated `.osm` input.

112
docs/refactor-plan.md Normal file
View File

@@ -0,0 +1,112 @@
# 重构施工计划:`osmassets` 包化P0P3
> 临时工作文档。P3 收尾后把结论并入 `docs/changelog.md`,本文件删除。
## 目标
把「从 OSM 生成 Blender / Cesium 资产」的逻辑从两个单体脚本里拆成可复用的库,使得:
- 新增一种 OSM 要素 = 新增一个 `features/*.py` + 注册一行,不改 `build()`
- 道路图层表、材质规格只有一份定义JS 侧与 Python 侧不再各存一份
- `export_cesium.py` 不再靠材质名字符串跟 `generate_scene.py` 对接
- 纯几何 / 解析逻辑脱离 `bpy`,可用系统 python 直接测
## 硬约束:严格产物一致
P0P3 全程 **不改变任何输出**。每期结束必须通过 parity 校验,任何差异都要么消除、要么在本文件里逐条记录原因。
已知缺陷(本轮**只记录、不修**
| # | 位置 | 现象 |
|---|---|---|
| D1 | `export_cesium.py:26,34,40,52` | `"Office White Metal Facade"` 四张表里都有,`generate_scene.py` 里已无此材质——死条目 |
| D2 | `scene-layers.js:15` vs `generate_scene.py:1017` | 同一批图层的颜色两侧各自手调,无一致性保证 |
| D3 | `generate_scene.py:788` | `tuft_density_wave` 注释仍在跟已删除的 hedge banding 作对比 |
## Parity 工具与基线
`outputs/` 已在 `.gitignore` 中,基线快照放 `outputs/_refactor-baseline/`,不入库。
| 工具 | 位置 | 作用 |
|---|---|---|
| 场景摘要 | `blender/tools/scene_digest.py` | 在 Blender 内打开 `.blend`,输出稳定 JSON对象名/顶点数/面数/材质槽/自定义属性、材质参数、场景属性 |
| GLB 摘要 | `scripts/glb-digest.js` | 纯 Node 读 GLB 的 JSON chunk输出 node/mesh/material 清单与 PBR 参数,附 buffer 字节长度 |
| 驱动 | `scripts/parity.sh <label>` | 跑 blender+cesium 阶段 → 收集 `SCENE_DONE` / `CESIUM_EXPORT_DONE` / 两份摘要 / 渲染 PNG 到 `outputs/_refactor-baseline/<label>/` |
**先做对照实验control**用未改动的代码连跑两次diff 两份摘要。这一步确定哪些字段天然不确定,把这些字段列入忽略名单。没做这步的 parity 校验是假的。
已完成,结论如下(`control-1` vs `control-2`,两区域):
- `.blend` **结构摘要两次完全一致** —— 这是主校验信号,可信
- `.blend` 文件 sha256 不一致:内嵌绝对路径 + 图片打包顺序随哈希表走
- 渲染 PNG sha256 不一致EEVEE 非位级可复现
- GLB 结构node / mesh / primitive / material / image两次完全一致但 accessor 数 399 vs 398、buffer 差 720 字节glTF 导出器会去重相同 accessor`smart_project` 的 UV 带浮点噪声,一次能去重一次不能
忽略名单(写在 `scripts/parity.js:IGNORED_PATHS`,附原因):`files.{blend,glb,render}.sha256``files.{glb,render}.bytes``glbDigest.{fileBytes,buffers,counts.accessors}`
保留比对的即真正的契约:`SCENE_DONE` / `CESIUM_EXPORT_DONE` 标记、`.blend` 全量结构摘要、GLB 的 node/mesh/material/image 结构、`<area>.json` 元数据。加上 `control-1``control-2` 两份基线已落盘。
样本区域:
- `nantaizi-lake-innovation-valley` — 主样本OSM + osm2streets GeoJSON 齐全
- `hanyang-block` — 次样本,只有 `intermediates` 产物,需先补跑一次 blender 阶段生成基线
## 分期
### P0 — 抽纯函数(行为零变化)
新建 `blender/osmassets/`,只搬运、不改逻辑:
| 目标文件 | 从 `generate_scene.py` 搬入 | 依赖 |
|---|---|---|
| `osm.py` | `tags` (94)、`parse_osm` (99)、`Projector` (140)、`parse_height` (506) | 无 bpy |
| `geom.py` | `geometry_rings` (406)、`feature_in_bounds` (418)、`clip_polygon` (426)、`sample_tree_row` (513)、`polygon_area` (688)、`point_in_polygon` (697)、`distance_to_ring` (712) | 无 bpy |
- `generate_scene.py` 顶部加 `sys.path` 引导(`--factory-startup``blender/` 不在 `sys.path`),改为 `from osmassets import ...`
- 新增 `blender/tests/test_geom.py``test_osm.py``unittest` 标准库,系统 `python3` 直接跑(本机 3.9,避免 3.10+ 语法)
- 验收:`python3 -m unittest discover blender/tests` 通过 + parity 全绿
### P1 — 单一定义源
新建 `blender/osmassets/catalog.py`
- `ROAD_LAYERS``id` / `blender_z` / `material_name` / `color`,替换 `generate_scene.py:1017``road_mats``1122``layer_z` 两份副本
- `MATERIAL_SPECS`:目前散在 `build()` 里的全部 `make_material` / `make_textured_material` 调用参数
- 新增一致性检查:读输出目录里已存在的 `osm2streets_scene_style.json`,比对图层 id 集合与顺序,不一致则打 warning**不**改颜色,改了就破坏 parity → 见 D2
验收parity 全绿;手动删一个图层 id 验证 warning 生效。
### P2 — 要素注册表
新建 `blender/osmassets/features/`,每种要素一个模块,导出 `SPEC`
```
water.py natural=water / water=lake
grass.py landuse=grass含 tuft 散布)
scrub.py natural=scrub
tree.py natural=tree 节点 + natural=tree_row + 两种树风格
building.py building=*(含 roof / windows
fountain.py amenity=fountain
roads.py osm2streets GeoJSON 图层 + highway 折线回退
```
- `scene.py::assemble()` 遍历注册表;`build()` 收缩为「解析 → assemble → 灯光相机 → 存盘渲染」
- 计数器改由注册表汇总,但 `SCENE_DONE``scene[...]` 的键名、顺序保持逐字不变
- if/elif 的**匹配顺序**是语义的一部分(`building` 分支在最后),注册表必须保序
验收parity 全绿 —— 这期风险最高,逐要素分次提交,每次单独跑 parity。
### P3 — 材质契约化
- `catalog.py` 的材质规格扩展出 cesium 段:`tint` / `metallic` / `base_color` / `emission`
- `generate_scene.py` 把规格写进材质自定义属性 `material["cesium_export"] = json.dumps(spec)`
- `export_cesium.py` 优先读自定义属性;读不到时回落到现有四张名字表(**原样保留,含 D1 死条目**),保证旧 `.blend` 仍能导出且 parity 成立
- `Tree Crown` 的程序化贴图特例保持不变
验收parity 全绿;另外用重构前生成的旧 `.blend` 跑一次导出,确认回落路径可用。
## 不在本轮范围
- 输出目标可插拔(整场景 / 每要素单独 GLB——原 P4
- `build-area.js` 里 390 行内联 HTML 与手写 glTF 的拆分——原 P4
- 上表 D1D3 的修复

8
package-lock.json generated
View File

@@ -1,12 +1,12 @@
{
"name": "osm2streets-qgis-workflow",
"version": "0.1.0",
"name": "osm-asset-pipeline",
"version": "0.3.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "osm2streets-qgis-workflow",
"version": "0.1.0",
"name": "osm-asset-pipeline",
"version": "0.3.0",
"dependencies": {
"osm2streets-js-node": "0.1.4"
}

View File

@@ -1,12 +1,14 @@
{
"name": "osm-gis-pipeline",
"version": "0.2.0",
"name": "osm-asset-pipeline",
"version": "0.3.0",
"private": true,
"type": "commonjs",
"scripts": {
"build": "node scripts/build-osm2streets-qgis.js"
"build": "node scripts/build-area.js",
"build:area": "node scripts/build-area.js",
"build:qgis": "node scripts/build-osm2streets-qgis.js"
},
"dependencies": {
"osm2streets-js-node": "0.1.4"
}
}
}

774
scripts/build-area.js Executable file
View File

@@ -0,0 +1,774 @@
#!/usr/bin/env node
const fs = require("fs");
const path = require("path");
const { spawnSync } = require("child_process");
const repoRoot = path.resolve(__dirname, "..");
const args = parseArgs(process.argv.slice(2));
const configPath = path.resolve(
args.config || path.join(repoRoot, "config", "areas", "nantaizi-lake-innovation-valley.json"),
);
const area = normalizeAreaConfig(readJson(configPath));
const requestedStages = args.stages
? splitList(args.stages)
: null;
const stages = resolveStages(area.stages, requestedStages);
// intermediates deletes and rebuilds the GeoPackage from OSM, which is exactly
// the manual work reimport exists to recover. Refuse the combination instead of
// silently letting one undo the other.
if (stages.intermediates && stages.reimport) {
throw new Error(
"Stages 'intermediates' and 'reimport' are mutually exclusive: " +
"intermediates rebuilds the GeoPackage from OSM and would discard the QGIS edits reimport reads back.",
);
}
console.log(`Area: ${area.id}`);
console.log(`Config: ${configPath}`);
console.log(`Output: ${area.outputs.areaDir}`);
if (stages.intermediates) {
buildIntermediates(area);
}
if (stages.reimport) {
reimportGpkg(area);
}
if (stages.blender) {
buildBlenderScene(area);
}
if (stages.cesium) {
exportCesium(area);
}
if (stages.preview) {
writeCesiumPreview(area);
}
console.log("Done.");
function parseArgs(argv) {
const out = {};
for (let i = 0; i < argv.length; i += 1) {
const arg = argv[i];
if (!arg.startsWith("--")) continue;
const key = arg.slice(2).replace(/-([a-z])/g, (_, c) => c.toUpperCase());
const next = argv[i + 1];
if (!next || next.startsWith("--")) {
out[key] = "true";
} else {
out[key] = next;
i += 1;
}
}
return out;
}
function readJson(file) {
if (!fs.existsSync(file)) {
throw new Error(`Config file not found: ${file}`);
}
return JSON.parse(fs.readFileSync(file, "utf8"));
}
function normalizeAreaConfig(raw) {
const id = requireText(raw.id, "id");
const input = path.resolve(requireText(raw.input, "input"));
if (!fs.existsSync(input)) {
throw new Error(`Input OSM XML not found: ${input}`);
}
const outputRoot = path.resolve(raw.outputRoot || path.join(repoRoot, "outputs"));
const outputOverrides = raw.outputs || {};
const areaDir = path.resolve(outputOverrides.areaDir || path.join(outputRoot, id));
const fileStem = outputOverrides.fileStem || id;
const outputs = {
areaDir,
geojsonDir: path.resolve(outputOverrides.geojsonDir || path.join(areaDir, "osm2streets_web_out")),
gpkg: path.resolve(outputOverrides.gpkg || path.join(areaDir, `${fileStem}.gpkg`)),
qgisProject: path.resolve(outputOverrides.qgisProject || path.join(areaDir, `${fileStem}.qgz`)),
qgisPreview: path.resolve(outputOverrides.qgisPreview || path.join(areaDir, `${fileStem}-preview.png`)),
blend: path.resolve(outputOverrides.blend || path.join(areaDir, `${fileStem}.blend`)),
render: path.resolve(outputOverrides.render || path.join(areaDir, `${fileStem}.png`)),
glb: path.resolve(outputOverrides.glb || path.join(areaDir, `${fileStem}.glb`)),
metadata: path.resolve(outputOverrides.metadata || path.join(areaDir, `${fileStem}.json`)),
cesiumPreview: path.resolve(
outputOverrides.cesiumPreview || path.join(areaDir, `${fileStem}-cesium-preview.html`),
),
vehicleRoute: path.resolve(outputOverrides.vehicleRoute || path.join(areaDir, `${fileStem}-vehicle-route.json`)),
vehicleModel: path.resolve(outputOverrides.vehicleModel || path.join(areaDir, `${fileStem}-vehicle-car.gltf`)),
pipelineDir: path.resolve(outputOverrides.pipelineDir || path.join(areaDir, "_pipeline")),
};
return {
id,
input,
outputRoot,
qgisApp: raw.qgisApp || "/Applications/QGIS.app",
blenderApp: raw.blenderApp || "/Applications/Blender.app",
stages: {
intermediates: raw.stages?.intermediates ?? raw.stages?.qgis ?? true,
blender: raw.stages?.blender ?? true,
cesium: raw.stages?.cesium ?? true,
reimport: false,
preview: false,
},
qgis: {
arrowScale: raw.qgis?.arrowScale ?? raw.arrowScale ?? 0.8,
arrowMergeTriangles: raw.qgis?.arrowMergeTriangles ?? raw.arrowMergeTriangles ?? true,
arrowOutlineSimplifyMeters: raw.qgis?.arrowOutlineSimplifyMeters ?? raw.arrowOutlineSimplifyMeters ?? 0.05,
intersectionCornerSourceMaxDimensionMeters: raw.qgis?.intersectionCornerSourceMaxDimensionMeters ?? raw.intersectionCornerSourceMaxDimensionMeters ?? 2.6,
clipPad: raw.qgis?.clipPad ?? raw.clipPad ?? 0.002,
canvasPad: raw.qgis?.canvasPad ?? raw.canvasPad ?? 0.001,
previewPad: raw.qgis?.previewPad ?? raw.previewPad ?? 0.0007,
canvasExtent: raw.qgis?.canvasExtent ?? raw.canvasExtent ?? null,
previewExtent: raw.qgis?.previewExtent ?? raw.previewExtent ?? null,
layerPrefix: raw.qgis?.layerPrefix ?? raw.layerPrefix ?? "osm2streets",
},
osm2streets: raw.osm2streets || {
debug_each_step: false,
dual_carriageway_experiment: false,
sidepath_zipping_experiment: false,
inferred_sidewalks: true,
osm2lanes: true,
},
blender: {
treeStyle: raw.blender?.treeStyle || "natural",
officeOverrides: raw.blender?.officeOverrides || raw.blender?.office_overrides || "",
},
outputs,
};
}
function requireText(value, key) {
if (typeof value !== "string" || value.trim() === "") {
throw new Error(`Missing config key: ${key}`);
}
return value;
}
function splitList(value) {
return String(value)
.split(",")
.map((item) => item.trim())
.filter(Boolean);
}
function resolveStages(defaults, requested) {
if (!requested) return defaults;
// 'reimport' is deliberately absent from 'all': it is a recovery step for
// hand-edited GeoPackages, never part of a full build.
const aliases = {
all: ["intermediates", "blender", "cesium"],
qgis: ["intermediates"],
osm2streets: ["intermediates"],
geojson: ["intermediates"],
intermediate: ["intermediates"],
intermediates: ["intermediates"],
reimport: ["reimport"],
gpkg: ["reimport"],
blender: ["blender"],
scene: ["blender"],
cesium: ["cesium"],
glb: ["cesium"],
preview: ["preview"],
html: ["preview"],
cesiumPreview: ["preview"],
};
const out = { intermediates: false, reimport: false, blender: false, cesium: false, preview: false };
for (const stage of requested) {
const mapped = aliases[stage];
if (!mapped) {
throw new Error(`Unknown stage '${stage}'. Use intermediates, reimport, blender, cesium, preview, or all.`);
}
for (const key of mapped) out[key] = true;
}
return out;
}
function writeDerivedConfig(area) {
fs.mkdirSync(area.outputs.pipelineDir, { recursive: true });
const derivedConfig = {
qgisApp: area.qgisApp,
input: area.input,
outDir: area.outputs.geojsonDir,
gpkg: area.outputs.gpkg,
project: area.outputs.qgisProject,
preview: area.outputs.qgisPreview,
arrowScale: area.qgis.arrowScale,
arrowMergeTriangles: area.qgis.arrowMergeTriangles,
arrowOutlineSimplifyMeters: area.qgis.arrowOutlineSimplifyMeters,
intersectionCornerSourceMaxDimensionMeters: area.qgis.intersectionCornerSourceMaxDimensionMeters,
clipPad: area.qgis.clipPad,
canvasPad: area.qgis.canvasPad,
previewPad: area.qgis.previewPad,
canvasExtent: area.qgis.canvasExtent,
previewExtent: area.qgis.previewExtent,
layerPrefix: area.qgis.layerPrefix,
osm2streets: area.osm2streets,
};
const derivedConfigPath = path.join(area.outputs.pipelineDir, "osm2streets-qgis.config.json");
fs.writeFileSync(derivedConfigPath, `${JSON.stringify(derivedConfig, null, 2)}\n`);
return derivedConfigPath;
}
function buildIntermediates(area) {
const derivedConfigPath = writeDerivedConfig(area);
console.log("Stage: intermediates (osm2streets GeoJSON + QGIS)");
runCommand(process.execPath, [
path.join(repoRoot, "scripts", "build-osm2streets-qgis.js"),
"--config",
derivedConfigPath,
], "intermediates");
}
function reimportGpkg(area) {
const derivedConfigPath = writeDerivedConfig(area);
console.log("Stage: reimport (GeoPackage -> GeoJSON)");
runCommand(process.execPath, [
path.join(repoRoot, "scripts", "reimport-gpkg.js"),
"--config",
derivedConfigPath,
], "reimport");
}
function buildBlenderScene(area) {
ensureFile(blenderExecutable(area), "Blender executable");
ensureFile(path.join(repoRoot, "blender", "generate_scene.py"), "Blender scene generator");
fs.mkdirSync(path.dirname(area.outputs.blend), { recursive: true });
fs.mkdirSync(path.dirname(area.outputs.render), { recursive: true });
const blenderArgs = [
"--background",
"--factory-startup",
"--python",
path.join(repoRoot, "blender", "generate_scene.py"),
"--",
"--osm",
area.input,
"--geojson",
area.outputs.geojsonDir,
"--output",
area.outputs.blend,
"--render",
area.outputs.render,
"--tree-style",
area.blender.treeStyle,
];
if (area.blender.officeOverrides) {
blenderArgs.push("--office-overrides", area.blender.officeOverrides);
}
console.log("Stage: blender");
runCommand(blenderExecutable(area), blenderArgs, "blender");
}
function exportCesium(area) {
ensureFile(blenderExecutable(area), "Blender executable");
ensureFile(area.outputs.blend, "Blend scene");
ensureFile(path.join(repoRoot, "blender", "export_cesium.py"), "Cesium exporter");
fs.mkdirSync(path.dirname(area.outputs.glb), { recursive: true });
fs.mkdirSync(path.dirname(area.outputs.metadata), { recursive: true });
console.log("Stage: cesium");
runCommand(blenderExecutable(area), [
"--background",
"--python",
path.join(repoRoot, "blender", "export_cesium.py"),
"--",
"--blend",
area.outputs.blend,
"--glb",
area.outputs.glb,
"--metadata",
area.outputs.metadata,
], "cesium");
writeCesiumPreview(area);
}
function blenderExecutable(area) {
return path.join(area.blenderApp, "Contents", "MacOS", "Blender");
}
function ensureFile(file, label) {
if (!fs.existsSync(file)) {
throw new Error(`${label} not found: ${file}`);
}
}
function runCommand(command, commandArgs, stage) {
const result = spawnSync(command, commandArgs, { stdio: "inherit" });
if (result.error) {
throw result.error;
}
if (result.status !== 0) {
const signal = result.signal ? ` signal=${result.signal}` : "";
throw new Error(`Stage '${stage}' failed with status=${result.status}${signal}`);
}
}
function writeCesiumPreview(area) {
ensureFile(area.outputs.glb, "Cesium GLB");
ensureFile(area.outputs.metadata, "Cesium metadata");
const htmlPath = area.outputs.cesiumPreview;
fs.mkdirSync(path.dirname(htmlPath), { recursive: true });
writeVehicleRoute(area);
writeVehicleModel(area);
writeCesiumPreviewSupportFiles(path.dirname(htmlPath));
const glbName = path.basename(area.outputs.glb);
const metadataName = path.basename(area.outputs.metadata);
const routeName = path.basename(area.outputs.vehicleRoute);
const vehicleModelName = path.basename(area.outputs.vehicleModel);
fs.writeFileSync(htmlPath, cesiumPreviewHtml(glbName, metadataName, routeName, vehicleModelName, area.id));
console.log(`Cesium preview: ${htmlPath}`);
}
function writeCesiumPreviewSupportFiles(outDir) {
for (const file of ["cesium-preview.css", "cesium-preview.js"]) {
const source = path.join(repoRoot, "scripts", "lib", file);
ensureFile(source, `Cesium preview support file '${file}'`);
fs.copyFileSync(source, path.join(outDir, file));
}
}
function writeVehicleRoute(area) {
const route = buildVehicleRoute(area.input);
fs.mkdirSync(path.dirname(area.outputs.vehicleRoute), { recursive: true });
fs.writeFileSync(area.outputs.vehicleRoute, `${JSON.stringify(route, null, 2)}\n`);
console.log(`Vehicle route: ${area.outputs.vehicleRoute}`);
}
function writeVehicleModel(area) {
const gltf = makeVehicleGltf();
fs.mkdirSync(path.dirname(area.outputs.vehicleModel), { recursive: true });
fs.writeFileSync(area.outputs.vehicleModel, `${JSON.stringify(gltf, null, 2)}\n`);
console.log(`Vehicle model: ${area.outputs.vehicleModel}`);
}
function buildVehicleRoute(osmPath) {
const xml = fs.readFileSync(osmPath, "utf8");
const bounds = osmBounds(xml);
const nodes = new Map();
for (const match of xml.matchAll(/<node\b([^>]*)>/g)) {
const attrs = xmlAttrs(match[1]);
if (!attrs.id || attrs.lon === undefined || attrs.lat === undefined) continue;
nodes.set(attrs.id, [Number(attrs.lon), Number(attrs.lat)]);
}
const segments = [];
for (const match of xml.matchAll(/<way\b([^>]*)>([\s\S]*?)<\/way>/g)) {
const attrs = xmlAttrs(match[1]);
const body = match[2];
const tags = {};
for (const tagMatch of body.matchAll(/<tag\b([^>]*)\/?>/g)) {
const tag = xmlAttrs(tagMatch[1]);
if (tag.k) tags[tag.k] = tag.v || "";
}
if (!isCruiseHighway(tags)) continue;
const coords = [];
for (const ndMatch of body.matchAll(/<nd\b([^>]*)\/?>/g)) {
const nd = xmlAttrs(ndMatch[1]);
const coord = nodes.get(nd.ref);
if (coord) coords.push(coord);
}
const runs = splitInBounds(compactCoords(coords), bounds);
let runIndex = 0;
for (const run of runs) {
const lengthMeters = routeLength(run);
if (lengthMeters < 20) continue;
runIndex += 1;
const laneOffsetMeters = 1.3;
const shiftedRun = offsetPolylineRight(run, laneOffsetMeters);
segments.push({
id: runIndex === 1 ? (attrs.id || `way-${segments.length + 1}`) : `${attrs.id || "way"}-${runIndex}`,
name: tags.name || tags.highway || "road",
highway: tags.highway || "",
oneWay: tags.oneway || "",
lengthMeters,
laneOffsetMeters,
coordinates: shiftedRun,
centerlineCoordinates: run,
});
}
}
segments.sort((a, b) => b.lengthMeters - a.lengthMeters);
return {
source: osmPath,
bounds,
generatedAt: new Date().toISOString(),
speedMetersPerSecond: 8.0,
loop: true,
segments,
};
}
function osmBounds(xml) {
const match = xml.match(/<bounds\b([^>]*)\/?>/);
if (!match) return null;
const attrs = xmlAttrs(match[1]);
const bounds = {
minLon: Number(attrs.minlon),
minLat: Number(attrs.minlat),
maxLon: Number(attrs.maxlon),
maxLat: Number(attrs.maxlat),
};
return Object.values(bounds).every(Number.isFinite) ? bounds : null;
}
function xmlAttrs(text) {
const attrs = {};
for (const match of text.matchAll(/([:\w-]+)\s*=\s*("([^"]*)"|'([^']*)')/g)) {
attrs[match[1]] = match[3] !== undefined ? match[3] : match[4];
}
return attrs;
}
function isCruiseHighway(tags) {
const highway = tags.highway || "";
if (!highway) return false;
if (tags.area === "yes") return false;
const blocked = new Set([
"footway", "path", "pedestrian", "steps", "cycleway", "service", "track",
"bridleway", "corridor", "elevator", "platform", "construction",
]);
return !blocked.has(highway);
}
function compactCoords(coords) {
const out = [];
for (const coord of coords) {
const last = out[out.length - 1];
if (!last || last[0] !== coord[0] || last[1] !== coord[1]) {
out.push(coord);
}
}
return out;
}
function offsetPolylineRight(coords, offsetMeters) {
if (coords.length < 2 || offsetMeters === 0) return coords;
const refLat = coords.reduce((sum, coord) => sum + coord[1], 0) / coords.length;
const metersPerLat = 111320.0;
const metersPerLon = 111320.0 * Math.cos(degreesToRadians(refLat));
const points = coords.map((coord) => ({
x: coord[0] * metersPerLon,
y: coord[1] * metersPerLat,
lon: coord[0],
lat: coord[1],
}));
return points.map((point, index) => {
const prev = points[Math.max(0, index - 1)];
const next = points[Math.min(points.length - 1, index + 1)];
let dx = next.x - prev.x;
let dy = next.y - prev.y;
const length = Math.hypot(dx, dy);
if (length < 0.001) return [point.lon, point.lat];
dx /= length;
dy /= length;
const rightX = dy;
const rightY = -dx;
return [
(point.x + rightX * offsetMeters) / metersPerLon,
(point.y + rightY * offsetMeters) / metersPerLat,
];
});
}
function splitInBounds(coords, bounds) {
if (!bounds) return [coords];
const runs = [];
let current = [];
for (const coord of coords) {
if (insideBounds(coord, bounds)) {
current.push(coord);
} else if (current.length) {
if (current.length >= 2) runs.push(current);
current = [];
}
}
if (current.length >= 2) runs.push(current);
return runs;
}
function insideBounds(coord, bounds) {
const pad = 0.00002;
return (
coord[0] >= bounds.minLon - pad &&
coord[0] <= bounds.maxLon + pad &&
coord[1] >= bounds.minLat - pad &&
coord[1] <= bounds.maxLat + pad
);
}
function routeLength(coords) {
let total = 0;
for (let i = 1; i < coords.length; i += 1) {
total += haversineMeters(coords[i - 1], coords[i]);
}
return total;
}
function haversineMeters(a, b) {
const radius = 6371008.8;
const lat1 = degreesToRadians(a[1]);
const lat2 = degreesToRadians(b[1]);
const dLat = degreesToRadians(b[1] - a[1]);
const dLon = degreesToRadians(b[0] - a[0]);
const sinLat = Math.sin(dLat / 2);
const sinLon = Math.sin(dLon / 2);
const h = sinLat * sinLat + Math.cos(lat1) * Math.cos(lat2) * sinLon * sinLon;
return 2 * radius * Math.asin(Math.min(1, Math.sqrt(h)));
}
function degreesToRadians(value) {
return value * Math.PI / 180;
}
function makeVehicleGltf() {
const meshes = [];
const nodes = [];
const bufferParts = [];
const bufferViews = [];
const accessors = [];
function align4(bytes) {
while (bytes.length % 4 !== 0) bytes.push(0);
}
function addBufferView(bytes, target) {
align4(bufferParts);
const offset = bufferParts.length;
bufferParts.push(...bytes);
const view = { buffer: 0, byteOffset: offset, byteLength: bytes.length };
if (target) view.target = target;
bufferViews.push(view);
return bufferViews.length - 1;
}
function floatBytes(values) {
const buffer = Buffer.alloc(values.length * 4);
values.forEach((value, index) => buffer.writeFloatLE(value, index * 4));
return Array.from(buffer);
}
function ushortBytes(values) {
const buffer = Buffer.alloc(values.length * 2);
values.forEach((value, index) => buffer.writeUInt16LE(value, index * 2));
return Array.from(buffer);
}
function addAccessor(bufferView, componentType, count, type, min, max) {
const accessor = { bufferView, componentType, count, type };
if (min) accessor.min = min;
if (max) accessor.max = max;
accessors.push(accessor);
return accessors.length - 1;
}
function addMesh(name, geometry, material) {
const positionView = addBufferView(floatBytes(geometry.positions), 34962);
const indexView = addBufferView(ushortBytes(geometry.indices), 34963);
const positionAccessor = addAccessor(
positionView,
5126,
geometry.positions.length / 3,
"VEC3",
geometry.min,
geometry.max,
);
const indexAccessor = addAccessor(indexView, 5123, geometry.indices.length, "SCALAR");
meshes.push({
name,
primitives: [{
attributes: { POSITION: positionAccessor },
indices: indexAccessor,
material,
}],
});
nodes.push({ name, mesh: meshes.length - 1 });
}
addMesh("body", cuboid(0, 0, 0.72, 4.6, 1.9, 0.9), 0);
addMesh("hood", cuboid(1.35, 0, 1.1, 1.25, 1.74, 0.38), 0);
addMesh("cabin", cuboid(-0.55, 0, 1.42, 1.75, 1.55, 0.82), 1);
addMesh("rear", cuboid(-1.65, 0, 1.08, 0.95, 1.78, 0.42), 0);
addMesh("front_windshield", cuboid(0.28, 0, 1.58, 0.12, 1.42, 0.58), 2);
addMesh("left_window", cuboid(-0.55, -0.82, 1.52, 1.25, 0.08, 0.48), 2);
addMesh("right_window", cuboid(-0.55, 0.82, 1.52, 1.25, 0.08, 0.48), 2);
for (const x of [-1.55, 1.45]) {
for (const y of [-1.02, 1.02]) {
addMesh(`wheel_${x}_${y}`, cylinderY(x, y, 0.46, 0.38, 0.32, 16), 3);
addMesh(`hub_${x}_${y}`, cylinderY(x, y, 0.46, 0.2, 0.34, 12), 4);
}
}
addMesh("left_headlight", cuboid(2.36, -0.48, 0.9, 0.08, 0.32, 0.16), 5);
addMesh("right_headlight", cuboid(2.36, 0.48, 0.9, 0.08, 0.32, 0.16), 5);
addMesh("left_tail", cuboid(-2.36, -0.55, 0.9, 0.08, 0.28, 0.16), 6);
addMesh("right_tail", cuboid(-2.36, 0.55, 0.9, 0.08, 0.28, 0.16), 6);
const buffer = Buffer.from(bufferParts);
return {
asset: { version: "2.0", generator: "osm-asset-pipeline vehicle preview" },
scene: 0,
scenes: [{ nodes: nodes.map((_, index) => index) }],
nodes,
meshes,
buffers: [{
byteLength: buffer.length,
uri: `data:application/octet-stream;base64,${buffer.toString("base64")}`,
}],
bufferViews,
accessors,
materials: [
material("paint red", [0.82, 0.05, 0.035, 1], 0.55, 0.25),
material("dark roof", [0.08, 0.08, 0.085, 1], 0.45, 0.35),
material("glass", [0.04, 0.12, 0.16, 0.82], 0.18, 0.08),
material("tire", [0.015, 0.014, 0.013, 1], 0.75, 0.65),
material("wheel hub", [0.72, 0.72, 0.68, 1], 0.35, 0.85),
material("headlight", [1.0, 0.92, 0.62, 1], 0.12, 0.0),
material("tail light", [0.95, 0.03, 0.03, 1], 0.25, 0.0),
],
};
}
function material(name, color, roughness, metallic) {
return {
name,
pbrMetallicRoughness: {
baseColorFactor: color,
roughnessFactor: roughness,
metallicFactor: metallic,
},
};
}
function cuboid(cx, lateral, up, sx, width, height) {
const x0 = cx - sx / 2;
const x1 = cx + sx / 2;
const y0 = up - height / 2;
const y1 = up + height / 2;
const z0 = lateral - width / 2;
const z1 = lateral + width / 2;
const positions = [
x0, y0, z0, x1, y0, z0, x1, y1, z0, x0, y1, z0,
x0, y0, z1, x1, y0, z1, x1, y1, z1, x0, y1, z1,
];
const indices = [
0, 1, 2, 0, 2, 3, 4, 6, 5, 4, 7, 6,
0, 4, 5, 0, 5, 1, 1, 5, 6, 1, 6, 2,
2, 6, 7, 2, 7, 3, 3, 7, 4, 3, 4, 0,
];
return { positions, indices, min: [x0, y0, z0], max: [x1, y1, z1] };
}
function cylinderY(cx, lateral, up, radius, width, segments) {
const positions = [];
const indices = [];
const z0 = lateral - width / 2;
const z1 = lateral + width / 2;
for (const z of [z0, z1]) {
positions.push(cx, up, z);
for (let i = 0; i < segments; i += 1) {
const angle = 2 * Math.PI * i / segments;
positions.push(cx + Math.cos(angle) * radius, up + Math.sin(angle) * radius, z);
}
}
const center0 = 0;
const center1 = segments + 1;
for (let i = 0; i < segments; i += 1) {
const a0 = center0 + 1 + i;
const b0 = center0 + 1 + ((i + 1) % segments);
const a1 = center1 + 1 + i;
const b1 = center1 + 1 + ((i + 1) % segments);
indices.push(center0, b0, a0);
indices.push(center1, a1, b1);
indices.push(a0, b0, b1, a0, b1, a1);
}
return {
positions,
indices,
min: [cx - radius, up - radius, z0],
max: [cx + radius, up + radius, z1],
};
}
function cesiumPreviewHtml(glbName, metadataName, routeName, vehicleModelName, areaId) {
const previewConfig = {
areaId,
glbName,
metadataName,
routeName,
vehicleModelName,
};
return `<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>${escapeHtml(areaId)} Cesium Preview</title>
<script src="https://cdn.jsdelivr.net/npm/cesium@1.121.1/Build/Cesium/Cesium.js"></script>
<link href="https://cdn.jsdelivr.net/npm/cesium@1.121.1/Build/Cesium/Widgets/widgets.css" rel="stylesheet">
<link href="cesium-preview.css" rel="stylesheet">
</head>
<body>
<div id="cesiumContainer"></div>
<div id="controls">
<div class="control-group">
<button id="toggleCruise">Pause</button>
<button id="toggleFollow">Follow</button>
<label>Vehicle <select id="vehicleSelect"></select></label>
<label>Speed <input id="speedControl" type="range" min="2" max="22" step="1" value="8"></label>
<span id="speedLabel">8 m/s</span>
</div>
<div class="control-group">
<label><input id="toggleScene" type="checkbox" checked> Scene</label>
<span id="assetToggles" class="control-subgroup"></span>
<label><input id="toggleRoutes" type="checkbox" checked> Routes</label>
<label><input id="toggleVehicles" type="checkbox" checked> Vehicles</label>
<label><input id="toggleFps" type="checkbox"> FPS</label>
<label><input id="toggleDiagnostics" type="checkbox" checked> Info</label>
</div>
<div class="control-group">
<button data-camera="overview">Overview</button>
<button data-camera="oblique">Oblique</button>
<button data-camera="detail">Detail</button>
<button data-camera="route">Route</button>
</div>
</div>
<div id="diagnostics">Loading diagnostics...</div>
<div id="status">Loading ${escapeHtml(glbName)}...</div>
<div id="loadingOverlay">
<div class="loading-card">
<div class="loading-spinner" aria-hidden="true"></div>
<div class="loading-copy">
<strong>Loading scene</strong>
<span>${escapeHtml(areaId)}</span>
</div>
<div class="loading-bar" aria-hidden="true"><span></span></div>
</div>
</div>
<script>window.OSM_ASSET_PREVIEW_CONFIG = ${escapeScriptJson(JSON.stringify(previewConfig))};</script>
<script src="cesium-preview.js"></script>
</body>
</html>
`;
}
function escapeHtml(value) {
return String(value)
.replaceAll("&", "&amp;")
.replaceAll("<", "&lt;")
.replaceAll(">", "&gt;")
.replaceAll('"', "&quot;");
}
function escapeScriptJson(value) {
return String(value)
.replaceAll("<", "\\u003c")
.replaceAll(">", "\\u003e")
.replaceAll("&", "\\u0026")
.replaceAll("\u2028", "\\u2028")
.replaceAll("\u2029", "\\u2029");
}

View File

@@ -5,6 +5,15 @@ const path = require("path");
const os = require("os");
const { execFileSync } = require("child_process");
const { JsStreetNetwork } = require("osm2streets-js-node");
const {
SCENE_LAYERS,
SCENE_FILE,
SCENE_STYLE_FILE,
layerFile,
mergeScene,
sceneStyle,
qgisRgba,
} = require("./lib/scene-layers");
const repoRoot = path.resolve(__dirname, "..");
const args = parseArgs(process.argv.slice(2));
@@ -14,12 +23,16 @@ const qgisApp = config.qgisApp;
const qgisMacOS = path.join(qgisApp, "Contents", "MacOS");
const qgisPython = path.join(qgisMacOS, "python3.12");
const ogr2ogr = path.join(qgisMacOS, "ogr2ogr");
const normalizeLaneArrowsScript = path.join(repoRoot, "scripts", "normalize-lane-arrows.py");
const inputPath = path.resolve(config.input);
const outDir = path.resolve(config.outDir);
const gpkgPath = path.resolve(config.gpkg);
const projectPath = path.resolve(config.project);
const previewPath = path.resolve(config.preview);
const arrowScale = Number(config.arrowScale);
const arrowMergeTriangles = config.arrowMergeTriangles !== false;
const arrowOutlineSimplifyMeters = Number(config.arrowOutlineSimplifyMeters ?? 0.05);
const intersectionCornerSourceMaxDimensionMeters = Number(config.intersectionCornerSourceMaxDimensionMeters ?? 2.6);
const clipPad = Number(config.clipPad);
const canvasPad = Number(config.canvasPad);
const previewPad = Number(config.previewPad);
@@ -28,6 +41,12 @@ const layerPrefix = config.layerPrefix || "osm2streets";
if (!Number.isFinite(arrowScale) || arrowScale <= 0) {
throw new Error(`Invalid arrowScale: ${config.arrowScale}`);
}
if (!Number.isFinite(arrowOutlineSimplifyMeters) || arrowOutlineSimplifyMeters < 0) {
throw new Error(`Invalid arrowOutlineSimplifyMeters: ${config.arrowOutlineSimplifyMeters}`);
}
if (!Number.isFinite(intersectionCornerSourceMaxDimensionMeters) || intersectionCornerSourceMaxDimensionMeters <= 0) {
throw new Error(`Invalid intersectionCornerSourceMaxDimensionMeters: ${config.intersectionCornerSourceMaxDimensionMeters}`);
}
for (const [key, value] of [["clipPad", clipPad], ["canvasPad", canvasPad], ["previewPad", previewPad]]) {
if (!Number.isFinite(value) || value < 0) {
throw new Error(`Invalid ${key}: ${config[key]}`);
@@ -41,6 +60,9 @@ for (const exe of [ogr2ogr, qgisPython]) {
throw new Error(`QGIS executable not found: ${exe}`);
}
}
if (!fs.existsSync(normalizeLaneArrowsScript)) {
throw new Error(`Lane-arrow normalizer not found: ${normalizeLaneArrowsScript}`);
}
fs.mkdirSync(outDir, { recursive: true });
fs.mkdirSync(path.dirname(gpkgPath), { recursive: true });
@@ -60,19 +82,22 @@ writeGeoJson(outDir, "lane_markings.geojson", network.toLaneMarkingsGeojson());
writeGeoJson(outDir, "intersection_markings.geojson", network.toIntersectionMarkingsGeojson());
fs.writeFileSync(path.join(outDir, "network.json"), network.toJson());
const split = splitLayers(outDir, arrowScale, osm);
writeJson(path.join(outDir, "road_surface.geojson"), split.roadSurface);
writeJson(path.join(outDir, "intersection_surface.geojson"), split.intersectionSurface);
writeJson(path.join(outDir, "sidewalks.geojson"), split.sidewalks);
writeJson(path.join(outDir, "lane_separators.geojson"), split.laneSeparators);
writeJson(path.join(outDir, "center_lines.geojson"), split.centerLines);
writeJson(path.join(outDir, "vehicle_stop_lines.geojson"), split.vehicleStopLines);
writeJson(path.join(outDir, "lane_arrows_webscale.geojson"), split.laneArrows);
writeJson(path.join(outDir, "sidewalk_corners.geojson"), split.sidewalkCorners);
writeJson(path.join(outDir, "crosswalks.geojson"), split.crosswalks);
writeJson(path.join(outDir, "osm2streets_scene.geojson"), mergedScene(split));
const split = splitLayers(
outDir,
arrowScale,
intersectionCornerSourceMaxDimensionMeters,
osm,
);
for (const layer of SCENE_LAYERS) {
writeJson(path.join(outDir, layerFile(layer)), split[layer.splitKey]);
}
if (arrowMergeTriangles) {
normalizeLaneArrows(path.join(outDir, "lane_arrows_webscale.geojson"), arrowOutlineSimplifyMeters);
split.laneArrows = JSON.parse(fs.readFileSync(path.join(outDir, "lane_arrows_webscale.geojson"), "utf8"));
}
writeJson(path.join(outDir, SCENE_FILE), mergeScene((layer) => split[layer.splitKey]));
fs.writeFileSync(
path.join(outDir, "osm2streets_scene_style.json"),
path.join(outDir, SCENE_STYLE_FILE),
JSON.stringify(sceneStyle(), null, 2),
);
@@ -80,15 +105,10 @@ if (fs.existsSync(gpkgPath)) {
fs.unlinkSync(gpkgPath);
}
const ogrEnv = qgisEnv();
importLayer(gpkgPath, path.join(outDir, "road_surface.geojson"), "road_surface", false, ogrEnv);
importLayer(gpkgPath, path.join(outDir, "intersection_surface.geojson"), "intersection_surface", true, ogrEnv);
importLayer(gpkgPath, path.join(outDir, "sidewalks.geojson"), "sidewalks", true, ogrEnv);
importLayer(gpkgPath, path.join(outDir, "sidewalk_corners.geojson"), "sidewalk_corners", true, ogrEnv);
importLayer(gpkgPath, path.join(outDir, "lane_separators.geojson"), "lane_separators", true, ogrEnv);
importLayer(gpkgPath, path.join(outDir, "center_lines.geojson"), "center_lines", true, ogrEnv);
importLayer(gpkgPath, path.join(outDir, "vehicle_stop_lines.geojson"), "vehicle_stop_lines", true, ogrEnv);
importLayer(gpkgPath, path.join(outDir, "lane_arrows_webscale.geojson"), "lane_arrows_webscale", true, ogrEnv);
importLayer(gpkgPath, path.join(outDir, "crosswalks.geojson"), "crosswalks", true, ogrEnv);
// First layer creates the GeoPackage; the rest append into it.
SCENE_LAYERS.forEach((layer, index) => {
importLayer(gpkgPath, path.join(outDir, layerFile(layer)), layer.id, index > 0, ogrEnv);
});
const qgisScript = path.join(outDir, "_create_qgis_project.py");
const previewFeature = split.crosswalks.features[0] || split.laneArrows.features[0] || split.roadSurface.features[0];
@@ -161,6 +181,9 @@ function loadConfig(file, cliArgs) {
project: "project",
preview: "preview",
arrowScale: "arrowScale",
arrowMergeTriangles: "arrowMergeTriangles",
arrowOutlineSimplifyMeters: "arrowOutlineSimplifyMeters",
intersectionCornerSourceMaxDimensionMeters: "intersectionCornerSourceMaxDimensionMeters",
clipPad: "clipPad",
pad: "clipPad",
canvasPad: "canvasPad",
@@ -381,58 +404,12 @@ function emptyCollection() {
return { type: "FeatureCollection", features: [] };
}
function mergedScene(split) {
const layers = [
["road_surface", 10, split.roadSurface],
["intersection_surface", 20, split.intersectionSurface],
["sidewalks", 30, split.sidewalks],
["sidewalk_corners", 40, split.sidewalkCorners],
["lane_separators", 50, split.laneSeparators],
["center_lines", 60, split.centerLines],
["crosswalks", 70, split.crosswalks],
["vehicle_stop_lines", 80, split.vehicleStopLines],
["lane_arrows_webscale", 90, split.laneArrows],
];
return {
type: "FeatureCollection",
features: layers.flatMap(([renderLayer, zIndex, collection]) => (
(collection.features || []).map((feature) => ({
...feature,
properties: {
...(feature.properties || {}),
render_layer: renderLayer,
z_index: zIndex,
},
}))
)),
};
}
function sceneStyle() {
return {
version: 1,
geometry: "polygon",
sortProperty: "z_index",
layerProperty: "render_layer",
layers: [
{ id: "road_surface", zIndex: 10, fill: "#2b2b28", outline: "#1e1e1c", outlineWidth: 0.04 },
{ id: "intersection_surface", zIndex: 20, fill: "#2b2b28", outline: "#1e1e1c", outlineWidth: 0.04 },
{ id: "sidewalks", zIndex: 30, fill: "#bebeb6", outline: "#9c9c94", outlineWidth: 0.025 },
{ id: "sidewalk_corners", zIndex: 40, fill: "#bebeb6", outline: "#9c9c94", outlineWidth: 0.025 },
{ id: "lane_separators", zIndex: 50, fill: "#eeeee6", outline: null, outlineWidth: 0 },
{ id: "center_lines", zIndex: 60, fill: "#f5be2a", outline: null, outlineWidth: 0 },
{ id: "crosswalks", zIndex: 70, fill: "#fffff6", outline: null, outlineWidth: 0 },
{ id: "vehicle_stop_lines", zIndex: 80, fill: "#fffff6", outline: null, outlineWidth: 0 },
{ id: "lane_arrows_webscale", zIndex: 90, fill: "#fffff6", outline: "#2b2b28", outlineWidth: 0.015 },
],
};
}
function splitLayers(dir, arrowScaleValue, osm) {
function splitLayers(dir, arrowScaleValue, maxCornerDimensionMeters, osm) {
const plain = JSON.parse(fs.readFileSync(path.join(dir, "plain.geojson"), "utf8"));
const lanePolygons = JSON.parse(fs.readFileSync(path.join(dir, "lane_polygons.geojson"), "utf8"));
const markings = JSON.parse(fs.readFileSync(path.join(dir, "lane_markings.geojson"), "utf8"));
const intersections = JSON.parse(fs.readFileSync(path.join(dir, "intersection_markings.geojson"), "utf8"));
const network = JSON.parse(fs.readFileSync(path.join(dir, "network.json"), "utf8"));
const crosswalkData = buildCrosswalks(osm);
const serviceWayIds = new Set([...osm.ways.values()]
.filter((way) => way.tags.highway === "service")
@@ -445,7 +422,7 @@ function splitLayers(dir, arrowScaleValue, osm) {
centerLines: emptyCollection(),
vehicleStopLines: crosswalkData.stopLines,
laneArrows: emptyCollection(),
sidewalkCorners: intersections,
sidewalkCorners: buildSidewalkCorners(intersections, plain, network, maxCornerDimensionMeters),
crosswalks: crosswalkData.stripes,
};
const serviceDrivingPolygons = [];
@@ -455,7 +432,6 @@ function splitLayers(dir, arrowScaleValue, osm) {
out.intersectionSurface.features.push(feature);
}
}
for (const feature of lanePolygons.features || []) {
const type = feature.properties?.type;
if (type === "Sidewalk" || type === "Footway") {
@@ -479,7 +455,9 @@ function splitLayers(dir, arrowScaleValue, osm) {
if (type === "vehicle stop line" && !isInAnyPolygon(feature, crosswalkData.stopLineExclusionZones, "intersects")) {
out.vehicleStopLines.features.push(feature);
}
if (type === "lane arrow" && !conflictsWithCrosswalk && !isInAnyPolygon(feature, serviceDrivingPolygons)) out.laneArrows.features.push(scaleFeature(feature, arrowScaleValue));
if (type === "lane arrow" && !conflictsWithCrosswalk && !isInAnyPolygon(feature, serviceDrivingPolygons)) {
out.laneArrows.features.push(scaleFeature(feature, arrowScaleValue));
}
}
return out;
@@ -490,6 +468,457 @@ function hasAnyWayId(value, ids) {
return values.some((id) => ids.has(Number(id)));
}
function filteredSidewalkCorners(intersections, maxDimensionMeters) {
const out = emptyCollection();
for (const feature of intersections.features || []) {
if (feature.properties?.type !== "sidewalk corner") continue;
const dimension = maxFeatureDimensionMeters(feature);
if (dimension === null || dimension > maxDimensionMeters) continue;
out.features.push(feature);
}
return out;
}
function buildSidewalkCorners(intersections, plain, network, maxDimensionMeters) {
const out = filteredSidewalkCorners(intersections, maxDimensionMeters);
const missing = synthesizeMissingSidewalkCorners(out, plain, network, maxDimensionMeters);
out.features.push(...missing);
const caps = synthesizeTJunctionSidewalkCaps(out, plain, network, maxDimensionMeters);
out.features.push(...caps);
return out;
}
function synthesizeMissingSidewalkCorners(existing, plain, network, maxDimensionMeters) {
const roadFeatures = new Map((plain.features || [])
.filter((feature) => feature.properties?.type === "road")
.map((feature) => [Number(feature.properties.id), feature]));
const intersectionFeatures = new Map((plain.features || [])
.filter((feature) => feature.properties?.type === "intersection")
.map((feature) => [Number(feature.properties.id), feature]));
const roads = new Map((network.roads || []).map(([id, road]) => [Number(id), road]));
const intersections = new Map((network.intersections || []).map(([id, intersection]) => [Number(id), intersection]));
const existingByIntersection = assignCornersToIntersections(existing.features || [], intersectionFeatures);
const synthesized = [];
for (const [intersectionId, intersection] of intersections.entries()) {
const intersectionFeature = intersectionFeatures.get(intersectionId);
if (!intersectionFeature || intersection.roads.length < 2) continue;
const edges = buildIntersectionEdges(intersection, roads, roadFeatures, intersectionFeature);
if (!edges.length) continue;
const qualifyingPairs = qualifyingCornerPairs(edges);
if (!qualifyingPairs.length) continue;
const current = existingByIntersection.get(intersectionId) || [];
const currentCenters = current.map((entry) => entry.center);
const missing = [];
for (const [one, two] of qualifyingPairs) {
const candidate = synthesizeCornerFeature(one, two, intersectionFeature, maxDimensionMeters);
if (!candidate) continue;
const candidateCenter = featureCenter(candidate);
if (!candidateCenter || !pointInPolygon(candidateCenter, intersectionFeature.geometry.coordinates)) continue;
const dimension = maxFeatureDimensionMeters(candidate);
if (dimension === null || dimension > maxDimensionMeters) continue;
if (polygonAreaMeters2(candidate) < 0.4) continue;
if (currentCenters.some((point) => pointDistance(point, candidateCenter) <= 0.6)) continue;
missing.push(candidate);
}
if (missing.length !== 1) continue;
synthesized.push(missing[0]);
}
return synthesized;
}
function synthesizeTJunctionSidewalkCaps(existing, plain, network, maxDimensionMeters) {
const roadFeatures = new Map((plain.features || [])
.filter((feature) => feature.properties?.type === "road")
.map((feature) => [Number(feature.properties.id), feature]));
const intersectionFeatures = new Map((plain.features || [])
.filter((feature) => feature.properties?.type === "intersection")
.map((feature) => [Number(feature.properties.id), feature]));
const roads = new Map((network.roads || []).map(([id, road]) => [Number(id), road]));
const intersections = new Map((network.intersections || []).map(([id, intersection]) => [Number(id), intersection]));
const existingByIntersection = assignCornersToIntersections(existing.features || [], intersectionFeatures);
const synthesized = [];
for (const [intersectionId, intersection] of intersections.entries()) {
const intersectionFeature = intersectionFeatures.get(intersectionId);
if (!intersectionFeature) continue;
if (intersectionFeature.properties?.intersection_kind !== "Intersection") continue;
if (new Set(intersection.roads || []).size !== 3) continue;
const edges = buildIntersectionEdges(intersection, roads, roadFeatures, intersectionFeature);
if (!edges.length) continue;
const current = existingByIntersection.get(intersectionId) || [];
const smallCurrent = current.filter((entry) => {
const dimension = maxFeatureDimensionMeters(entry.feature);
return Number.isFinite(dimension) && dimension <= maxDimensionMeters;
});
if (smallCurrent.length !== 2) continue;
const candidates = [];
for (const [one, two] of qualifyingCornerPairs(edges)) {
const candidate = synthesizeCornerFeature(one, two, intersectionFeature, 100);
if (!candidate) continue;
const center = featureCenter(candidate);
const dimension = maxFeatureDimensionMeters(candidate);
const area = polygonAreaMeters2(candidate);
if (!center || !Number.isFinite(dimension) || !Number.isFinite(area)) continue;
candidates.push({ feature: candidate, center, dimension, area });
}
// For a true T-junction, the remaining large candidate is the sidewalk "cap"
// opposite the side street, not another curb-return corner.
const caps = candidates.filter(({ dimension, area, center }) => (
dimension > maxDimensionMeters &&
dimension <= 12 &&
area >= 6 &&
area <= 20 &&
pointInPolygon(center, intersectionFeature.geometry.coordinates) &&
!smallCurrent.some((entry) => pointDistance(entry.center, center) <= 1)
));
if (caps.length !== 1) continue;
caps[0].feature.properties.source = "fallback_t_cap";
synthesized.push(caps[0].feature);
}
return synthesized;
}
function assignCornersToIntersections(features, intersectionFeatures) {
const out = new Map();
for (const feature of features) {
const point = featureCenter(feature);
if (!point) continue;
for (const [intersectionId, intersectionFeature] of intersectionFeatures.entries()) {
if (!pointInPolygon(point, intersectionFeature.geometry.coordinates)) continue;
const bucket = out.get(intersectionId) || [];
bucket.push({
feature,
center: point,
});
out.set(intersectionId, bucket);
break;
}
}
return out;
}
function buildIntersectionEdges(intersection, roads, roadFeatures, intersectionFeature) {
const edges = [];
for (const roadId of intersection.roads || []) {
const road = roads.get(Number(roadId));
const roadFeature = roadFeatures.get(Number(roadId));
if (!road || !roadFeature) return [];
const geometry = roadEndpointGeometry(road, roadFeature, intersectionFeature, intersection.id);
if (!geometry) return [];
const first = road.dst_i === intersection.id
? makeRoadEdge(road, geometry, "right")
: makeRoadEdge(road, geometry, "left");
const second = road.dst_i === intersection.id
? makeRoadEdge(road, geometry, "left")
: makeRoadEdge(road, geometry, "right");
if (!first || !second) return [];
edges.push(first, second);
}
return edges;
}
function qualifyingCornerPairs(edges) {
if (!edges.length) return [];
const loop = [...edges, edges[0]];
const pairs = [];
for (let i = 0; i < loop.length - 1; i += 1) {
const one = loop[i];
const two = loop[i + 1];
if (one.roadId === two.roadId) continue;
if (!isWalkableOuterLane(one.laneType) || !isWalkableOuterLane(two.laneType)) continue;
if (one.laneCount === 1 || two.laneCount === 1) continue;
pairs.push([one, two]);
}
return pairs;
}
function isWalkableOuterLane(type) {
return type === "Sidewalk" || type === "Shoulder";
}
function roadEndpointGeometry(road, roadFeature, intersectionFeature, intersectionId) {
const ring = normalizedRing(roadFeature.geometry.coordinates?.[0]);
if (ring.length !== 4) return null;
const shortEdges = shortEdgePairs(ring);
if (!shortEdges) return null;
const intersectionCenter = ringCenter(intersectionFeature.geometry.coordinates[0]);
const candidates = shortEdges.map(([a, b]) => {
const near = [ring[a], ring[b]];
return {
pair: [a, b],
center: midpoint(near[0], near[1]),
distance: pointDistanceMeters(midpoint(near[0], near[1]), intersectionCenter, metersForLat(intersectionCenter[1])),
};
});
candidates.sort((a, b) => a.distance - b.distance);
const nearPair = candidates[0].pair;
const farPair = candidates[1].pair;
const nearPoints = nearPair.map((idx) => ring[idx]);
const farPoints = farPair.map((idx) => ring[idx]);
const nearCenter = midpoint(nearPoints[0], nearPoints[1]);
const farCenter = midpoint(farPoints[0], farPoints[1]);
const roadDirection = road.src_i === intersectionId
? normalizeLonLatVector([farCenter[0] - nearCenter[0], farCenter[1] - nearCenter[1]], nearCenter[1])
: normalizeLonLatVector([nearCenter[0] - farCenter[0], nearCenter[1] - farCenter[1]], nearCenter[1]);
if (!roadDirection) return null;
const correspondences = nearPair.map((idx) => {
const farIdx = farPair.find((candidate) => circularIndexDistance(idx, candidate, ring.length) === 1);
return farIdx === undefined ? null : [ring[idx], ring[farIdx]];
});
if (correspondences.some((pair) => !pair)) return null;
const classified = correspondences.map(([nearPoint, farPoint]) => ({
near: nearPoint,
far: farPoint,
cross: signedSide(roadDirection, nearCenter, nearPoint, nearCenter[1]),
})).sort((a, b) => a.cross - b.cross);
return {
nearCenter,
nearLeft: classified[1].near,
farLeft: classified[1].far,
nearRight: classified[0].near,
farRight: classified[0].far,
};
}
function shortEdgePairs(ring) {
const lengths = ring.map((point, index) => lineLengthMeters(point, ring[(index + 1) % ring.length]));
const optionA = lengths[0] + lengths[2];
const optionB = lengths[1] + lengths[3];
if (!Number.isFinite(optionA) || !Number.isFinite(optionB)) return null;
return optionA <= optionB
? [[0, 1], [2, 3]]
: [[1, 2], [3, 0]];
}
function circularIndexDistance(a, b, size) {
const distance = Math.abs(a - b);
return Math.min(distance, size - distance);
}
function makeRoadEdge(road, geometry, side) {
const lane = side === "left"
? road.lane_specs_ltr?.[0]
: road.lane_specs_ltr?.[road.lane_specs_ltr.length - 1];
if (!lane) return null;
const outerNear = side === "left" ? geometry.nearLeft : geometry.nearRight;
const outerFar = side === "left" ? geometry.farLeft : geometry.farRight;
const oppositeNear = side === "left" ? geometry.nearRight : geometry.nearLeft;
const oppositeFar = side === "left" ? geometry.farRight : geometry.farLeft;
const widthMeters = Number(lane.width) / 10000;
const innerNear = moveTowards(outerNear, oppositeNear, widthMeters);
const innerFar = moveTowards(outerFar, oppositeFar, widthMeters);
return {
roadId: road.id,
laneType: lane.lt,
laneCount: road.lane_specs_ltr?.length || 0,
outerNear,
innerNear,
innerFar,
};
}
function synthesizeCornerFeature(one, two, intersectionFeature, maxDimensionMeters) {
const ring = normalizedRing(intersectionFeature.geometry.coordinates?.[0]);
const slice = shorterRingSliceBetween(ring, one.outerNear, two.outerNear);
if (!slice || slice.length < 2) return null;
const meetPoint = lineIntersection(one.innerFar, one.innerNear, two.innerFar, two.innerNear);
const points = dedupeSequentialPoints([
...slice,
two.innerNear,
...(meetPoint && pointInPolygon(meetPoint, intersectionFeature.geometry.coordinates) ? [meetPoint] : []),
one.innerNear,
slice[0],
]);
if (points.length < 4) return null;
const feature = {
type: "Feature",
properties: {
type: "sidewalk corner",
source: "fallback",
},
geometry: {
type: "Polygon",
coordinates: [points],
},
};
const dimension = maxFeatureDimensionMeters(feature);
if (dimension === null || dimension > maxDimensionMeters) return null;
return feature;
}
function shorterRingSliceBetween(ring, start, end) {
if (!ring.length) return null;
const startIndex = nearestRingPointIndex(ring, start, 0.8);
const endIndex = nearestRingPointIndex(ring, end, 0.8);
if (startIndex === null || endIndex === null) return null;
if (startIndex === endIndex) return [ring[startIndex]];
const forward = walkRing(ring, startIndex, endIndex, 1);
const backward = walkRing(ring, startIndex, endIndex, -1);
return pathLengthMeters(forward) <= pathLengthMeters(backward) ? forward : backward;
}
function walkRing(ring, startIndex, endIndex, direction) {
const out = [ring[startIndex]];
let index = startIndex;
while (index !== endIndex) {
index = (index + direction + ring.length) % ring.length;
out.push(ring[index]);
}
return out;
}
function pathLengthMeters(points) {
let total = 0;
for (let i = 1; i < points.length; i += 1) total += lineLengthMeters(points[i - 1], points[i]);
return total;
}
function lineIntersection(a1, a2, b1, b2) {
const originLat = (a1[1] + a2[1] + b1[1] + b2[1]) / 4;
const meters = metersForLat(originLat);
const ax1 = 0;
const ay1 = 0;
const ax2 = (a2[0] - a1[0]) * meters.lon;
const ay2 = (a2[1] - a1[1]) * meters.lat;
const bx1 = (b1[0] - a1[0]) * meters.lon;
const by1 = (b1[1] - a1[1]) * meters.lat;
const bx2 = (b2[0] - a1[0]) * meters.lon;
const by2 = (b2[1] - a1[1]) * meters.lat;
const denominator = (ax2 - ax1) * (by2 - by1) - (ay2 - ay1) * (bx2 - bx1);
if (Math.abs(denominator) < 1e-9) return null;
const ua = ((bx2 - bx1) * (ay1 - by1) - (by2 - by1) * (ax1 - bx1)) / denominator;
return [
a1[0] + ((ax1 + ua * (ax2 - ax1)) / meters.lon),
a1[1] + ((ay1 + ua * (ay2 - ay1)) / meters.lat),
];
}
function normalizedRing(ring) {
if (!Array.isArray(ring) || ring.length < 4) return [];
const out = ring.map((point) => [point[0], point[1]]);
if (pointDistance(out[0], out[out.length - 1]) <= 0.02) out.pop();
return out;
}
function nearestRingPointIndex(ring, point, maxDistanceMeters) {
let bestIndex = null;
let bestDistance = Infinity;
for (let i = 0; i < ring.length; i += 1) {
const distance = pointDistance(ring[i], point);
if (distance < bestDistance) {
bestDistance = distance;
bestIndex = i;
}
}
return bestDistance <= maxDistanceMeters ? bestIndex : null;
}
function nearestRingPoint(ring, point, maxDistanceMeters) {
const normalized = normalizedRing(ring);
const index = nearestRingPointIndex(normalized, point, maxDistanceMeters);
return index === null ? null : normalized[index];
}
function dedupeSequentialPoints(points, toleranceMeters = 0.02) {
const out = [];
for (const point of points) {
if (!out.length || pointDistance(out[out.length - 1], point) > toleranceMeters) out.push(point);
}
if (out.length >= 2 && pointDistance(out[0], out[out.length - 1]) > toleranceMeters) out.push(out[0]);
return out;
}
function dedupePointList(points, toleranceMeters) {
const out = [];
for (const point of points) {
if (out.some((other) => pointDistance(point, other) <= toleranceMeters)) continue;
out.push(point);
}
return out;
}
function midpoint(a, b) {
return [(a[0] + b[0]) / 2, (a[1] + b[1]) / 2];
}
function moveTowards(from, to, distanceMeters) {
const meters = metersForLat((from[1] + to[1]) / 2);
const unit = normalizeMetersVector([to[0] - from[0], to[1] - from[1]], meters);
if (!unit) return from;
return addMeters(from, unit, distanceMeters, meters);
}
function normalizeLonLatVector([dxLon, dyLat], lat) {
return normalizeMetersVector([dxLon, dyLat], metersForLat(lat));
}
function signedSide(direction, origin, point, lat) {
const meters = metersForLat(lat);
const dx = (point[0] - origin[0]) * meters.lon;
const dy = (point[1] - origin[1]) * meters.lat;
return direction[0] * dy - direction[1] * dx;
}
function pointDistance(a, b) {
return lineLengthMeters(a, b);
}
function lineLengthMeters(a, b) {
const meters = metersForLat((a[1] + b[1]) / 2);
return Math.hypot((a[0] - b[0]) * meters.lon, (a[1] - b[1]) * meters.lat);
}
function ringCenter(ring) {
const points = normalizedRing(ring);
const xs = points.map((point) => point[0]);
const ys = points.map((point) => point[1]);
return [(Math.min(...xs) + Math.max(...xs)) / 2, (Math.min(...ys) + Math.max(...ys)) / 2];
}
function polygonAreaMeters2(feature) {
const ring = normalizedRing(feature.geometry?.coordinates?.[0]);
if (ring.length < 3) return 0;
const meters = metersForLat(ring.reduce((sum, point) => sum + point[1], 0) / ring.length);
let area = 0;
for (let i = 0; i < ring.length; i += 1) {
const a = ring[i];
const b = ring[(i + 1) % ring.length];
area += (a[0] * meters.lon) * (b[1] * meters.lat) - (b[0] * meters.lon) * (a[1] * meters.lat);
}
return Math.abs(area) / 2;
}
function maxFeatureDimensionMeters(feature) {
const points = [];
collectCoords(feature.geometry?.coordinates, points);
if (points.length === 0) return null;
const lat = points.reduce((sum, point) => sum + point[1], 0) / points.length;
const meters = metersForLat(lat);
const xs = points.map((point) => point[0]);
const ys = points.map((point) => point[1]);
const width = (Math.max(...xs) - Math.min(...xs)) * meters.lon;
const height = (Math.max(...ys) - Math.min(...ys)) * meters.lat;
return Math.max(width, height);
}
function polygonRings(geometry) {
if (!geometry?.coordinates) return [];
if (geometry.type === "Polygon") return [geometry.coordinates];
@@ -549,6 +978,15 @@ function representativePoint(feature) {
return coords[Math.floor(coords.length / 2)];
}
function featureCenter(feature) {
const coords = [];
collectCoords(feature.geometry?.coordinates, coords);
if (!coords.length) return null;
const xs = coords.map((point) => point[0]);
const ys = coords.map((point) => point[1]);
return [(Math.min(...xs) + Math.max(...xs)) / 2, (Math.min(...ys) + Math.max(...ys)) / 2];
}
function pointInPolygon(point, rings) {
if (!rings?.length || !pointInRing(point, rings[0])) return false;
return !rings.slice(1).some((ring) => pointInRing(point, ring));
@@ -846,6 +1284,26 @@ function scaleCoords(obj, cx, cy, scale) {
return obj;
}
function normalizeLaneArrows(geojsonPath, outlineSimplifyMeters) {
execFileSync(qgisPython, [
normalizeLaneArrowsScript,
"--input", geojsonPath,
"--outline-simplify-meters", String(outlineSimplifyMeters),
], {
stdio: "inherit",
env: {
...process.env,
...qgisEnv(),
QT_QPA_PLATFORM: "offscreen",
PYTHONHOME: path.join(qgisApp, "Contents", "Frameworks"),
PYTHONPATH: [
path.join(qgisApp, "Contents", "Resources", "python"),
path.join(qgisApp, "Contents", "Resources", "python", "plugins"),
].join(path.delimiter),
},
});
}
function importLayer(gpkg, source, layerName, update, env) {
const args = ["-f", "GPKG"];
if (update) args.push("-update", "-overwrite");
@@ -853,6 +1311,16 @@ function importLayer(gpkg, source, layerName, update, env) {
execFileSync(ogr2ogr, args, { stdio: "inherit", env: { ...process.env, ...env } });
}
function qgisLayerSpecs() {
return SCENE_LAYERS.map((layer) => ({
id: layer.id,
title: layer.title,
fill: qgisRgba(layer.fill),
outline: qgisRgba(layer.outline, layer.outlineAlpha ?? 255),
outlineWidth: String(layer.outlineWidth),
}));
}
function makeQgisScript(options) {
return `
from pathlib import Path
@@ -877,6 +1345,7 @@ PROJECT_PATH = ${JSON.stringify(options.projectPath)}
PREVIEW_PATH = ${JSON.stringify(options.previewPath)}
PREVIEW_EXTENT = [${options.previewExtent.split(",").map(Number).join(", ")}]
LAYER_PREFIX = ${JSON.stringify(options.layerPrefix || "osm2streets")}
LAYER_SPECS = ${JSON.stringify(qgisLayerSpecs(), null, 4)}
def fill_symbol(color, outline="0,0,0,0", outline_width="0"):
return QgsFillSymbol.createSimple({
@@ -905,17 +1374,16 @@ project.setCrs(QgsCoordinateReferenceSystem("EPSG:4326"))
project.setPresetHomePath(str(Path(PROJECT_PATH).parent))
layers = {
"road_surface": make_layer("road_surface", f"{LAYER_PREFIX} road surface", "43,43,40,255", "30,30,28,255", "0.04"),
"intersection_surface": make_layer("intersection_surface", f"{LAYER_PREFIX} intersection surface", "43,43,40,255", "30,30,28,255", "0.04"),
"sidewalks": make_layer("sidewalks", f"{LAYER_PREFIX} sidewalks", "190,190,182,255", "156,156,148,255", "0.025"),
"sidewalk_corners": make_layer("sidewalk_corners", f"{LAYER_PREFIX} sidewalk corners", "190,190,182,255", "156,156,148,255", "0.025"),
"crosswalks": make_layer("crosswalks", f"{LAYER_PREFIX} crosswalks", "255,255,246,255"),
"lane_separators": make_layer("lane_separators", f"{LAYER_PREFIX} lane separators", "238,238,230,255"),
"center_lines": make_layer("center_lines", f"{LAYER_PREFIX} center lines", "245,190,42,255"),
"vehicle_stop_lines": make_layer("vehicle_stop_lines", f"{LAYER_PREFIX} vehicle stop lines", "255,255,246,255"),
"lane_arrows": make_layer("lane_arrows_webscale", f"{LAYER_PREFIX} lane arrows", "255,255,246,255", "43,43,40,200", "0.015"),
spec["id"]: make_layer(
spec["id"],
f"{LAYER_PREFIX} {spec['title']}",
spec["fill"],
spec["outline"],
spec["outlineWidth"],
)
for spec in LAYER_SPECS
}
draw_order = ["road_surface", "intersection_surface", "sidewalks", "sidewalk_corners", "lane_separators", "center_lines", "crosswalks", "vehicle_stop_lines", "lane_arrows"]
draw_order = [spec["id"] for spec in LAYER_SPECS]
for key in draw_order:
project.addMapLayer(layers[key], False)
root = project.layerTreeRoot()

121
scripts/glb-digest.js Normal file
View File

@@ -0,0 +1,121 @@
#!/usr/bin/env node
"use strict";
// Structural digest of a GLB, for the osmassets refactor parity check.
//
// Byte-comparing the GLB is too strict: Blender packs images in hash-map order
// and the buffer padding shifts with it, so two runs of identical code can
// differ. This reads the glTF JSON chunk instead and reports the parts that
// carry meaning downstream in Cesium — node/mesh/material identity and PBR
// values — plus buffer lengths as a coarse size check.
//
// node scripts/glb-digest.js <file.glb> [--out digest.json]
const fs = require("fs");
const path = require("path");
function readGlbJson(file) {
const buffer = fs.readFileSync(file);
if (buffer.length < 12 || buffer.readUInt32LE(0) !== 0x46546c67) {
throw new Error(`Not a GLB (bad magic): ${file}`);
}
const total = buffer.readUInt32LE(8);
let offset = 12;
while (offset + 8 <= Math.min(total, buffer.length)) {
const chunkLength = buffer.readUInt32LE(offset);
const chunkType = buffer.readUInt32LE(offset + 4);
const start = offset + 8;
if (chunkType === 0x4e4f534a) {
return JSON.parse(buffer.slice(start, start + chunkLength).toString("utf8"));
}
offset = start + chunkLength;
}
throw new Error(`No JSON chunk found in ${file}`);
}
function round(value) {
if (typeof value === "number") return Number(value.toFixed(6));
if (Array.isArray(value)) return value.map(round);
return value;
}
function materialDigest(material) {
const pbr = material.pbrMetallicRoughness || {};
return {
name: material.name || null,
baseColorFactor: round(pbr.baseColorFactor || null),
metallicFactor: round(pbr.metallicFactor ?? null),
roughnessFactor: round(pbr.roughnessFactor ?? null),
hasBaseColorTexture: Boolean(pbr.baseColorTexture),
hasNormalTexture: Boolean(material.normalTexture),
emissiveFactor: round(material.emissiveFactor || null),
emissiveStrength: round(
material.extensions?.KHR_materials_emissive_strength?.emissiveStrength ?? null,
),
alphaMode: material.alphaMode || null,
doubleSided: material.doubleSided ?? null,
};
}
function digest(file) {
const gltf = readGlbJson(file);
const meshes = (gltf.meshes || []).map((mesh) => ({
name: mesh.name || null,
primitives: (mesh.primitives || []).map((primitive) => ({
material: primitive.material ?? null,
attributes: Object.keys(primitive.attributes || {}).sort(),
// Vertex/index counts live on the accessors; they are the real geometry
// fingerprint and stay stable regardless of buffer layout.
count: gltf.accessors?.[primitive.attributes?.POSITION]?.count ?? null,
indices: gltf.accessors?.[primitive.indices]?.count ?? null,
})),
}));
return {
file: path.basename(file),
fileBytes: fs.statSync(file).size,
counts: {
nodes: (gltf.nodes || []).length,
meshes: meshes.length,
materials: (gltf.materials || []).length,
images: (gltf.images || []).length,
accessors: (gltf.accessors || []).length,
},
extensionsUsed: (gltf.extensionsUsed || []).slice().sort(),
buffers: (gltf.buffers || []).map((buffer) => buffer.byteLength),
nodes: (gltf.nodes || [])
.map((node) => ({
name: node.name || null,
mesh: node.mesh ?? null,
translation: round(node.translation || null),
rotation: round(node.rotation || null),
scale: round(node.scale || null),
extras: node.extras ?? null,
}))
.sort((a, b) => String(a.name).localeCompare(String(b.name))),
meshes: meshes.sort((a, b) => String(a.name).localeCompare(String(b.name))),
materials: (gltf.materials || [])
.map(materialDigest)
.sort((a, b) => String(a.name).localeCompare(String(b.name))),
images: (gltf.images || [])
.map((image) => ({ name: image.name || null, mimeType: image.mimeType || null }))
.sort((a, b) => String(a.name).localeCompare(String(b.name))),
};
}
const argv = process.argv.slice(2);
const file = argv.find((arg) => !arg.startsWith("--"));
if (!file) {
console.error("usage: node scripts/glb-digest.js <file.glb> [--out digest.json]");
process.exit(1);
}
const outIndex = argv.indexOf("--out");
const result = digest(path.resolve(file));
const text = `${JSON.stringify(result, null, 2)}\n`;
if (outIndex >= 0 && argv[outIndex + 1]) {
const out = path.resolve(argv[outIndex + 1]);
fs.mkdirSync(path.dirname(out), { recursive: true });
fs.writeFileSync(out, text);
console.log(`GLB digest: ${out}`);
} else {
process.stdout.write(text);
}

View File

@@ -0,0 +1,230 @@
html,
body,
#cesiumContainer {
width: 100%;
height: 100%;
margin: 0;
overflow: hidden;
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
background: #d9e0e2;
}
#cesiumContainer {
opacity: 0;
transition: opacity 180ms ease-out;
}
body.scene-ready #cesiumContainer {
opacity: 1;
}
#controls,
#diagnostics,
#status {
position: absolute;
z-index: 1;
border-radius: 4px;
background: rgba(20, 24, 28, 0.82);
color: #fff;
font-size: 12px;
line-height: 1.45;
box-shadow: 0 8px 24px rgba(0, 0, 0, 0.22);
}
#loadingOverlay {
position: absolute;
inset: 0;
z-index: 3;
display: grid;
place-items: center;
background: #d9e0e2;
color: #1b2528;
font-size: 13px;
letter-spacing: 0;
transition: opacity 220ms ease-out, visibility 220ms ease-out;
}
.loading-card {
display: grid;
grid-template-columns: 32px minmax(180px, 260px);
gap: 12px;
align-items: center;
padding: 14px 16px;
border: 1px solid rgba(70, 86, 90, 0.16);
border-radius: 6px;
background: rgba(244, 248, 248, 0.88);
box-shadow: 0 12px 34px rgba(54, 68, 72, 0.20);
}
.loading-spinner {
width: 28px;
height: 28px;
border: 3px solid rgba(42, 67, 72, 0.18);
border-top-color: #147d8a;
border-radius: 50%;
animation: loading-spin 840ms linear infinite;
}
.loading-copy {
display: grid;
gap: 3px;
}
.loading-copy strong {
font-size: 13px;
font-weight: 650;
}
#loadingOverlay span {
color: #526064;
}
.loading-bar {
grid-column: 1 / -1;
position: relative;
height: 3px;
overflow: hidden;
border-radius: 999px;
background: rgba(42, 67, 72, 0.12);
}
.loading-bar span {
position: absolute;
inset: 0 auto 0 0;
width: 42%;
border-radius: inherit;
background: #147d8a;
animation: loading-bar 1.2s ease-in-out infinite;
}
@keyframes loading-spin {
to {
transform: rotate(360deg);
}
}
@keyframes loading-bar {
0% {
transform: translateX(-110%);
}
55% {
transform: translateX(70%);
}
100% {
transform: translateX(250%);
}
}
body.scene-ready #loadingOverlay {
visibility: hidden;
opacity: 0;
}
body.scene-error #loadingOverlay {
color: #5b1414;
}
body.scene-error .loading-spinner {
border-color: rgba(120, 30, 30, 0.18);
border-top-color: #9f2f2f;
animation-play-state: paused;
}
body.scene-error .loading-bar span {
background: #9f2f2f;
animation-play-state: paused;
}
#controls {
top: 12px;
left: 12px;
display: flex;
flex-direction: column;
gap: 6px;
padding: 8px;
}
#status {
left: 12px;
bottom: 12px;
max-width: 560px;
padding: 8px 10px;
}
#diagnostics {
right: 12px;
top: 12px;
min-width: 280px;
max-width: 360px;
padding: 10px 12px;
white-space: pre-line;
}
#controls button,
#controls select {
height: 28px;
border: 0;
border-radius: 4px;
}
#controls button {
padding: 0 10px;
background: #f2f5f7;
color: #111;
cursor: pointer;
}
#controls button:active:not(:disabled) {
transform: translateY(1px);
}
#controls :disabled {
opacity: 0.45;
cursor: default;
}
#controls label:has(:disabled) {
opacity: 0.45;
}
#controls input[type="range"] {
width: 120px;
}
#controls label {
display: inline-flex;
gap: 5px;
align-items: center;
white-space: nowrap;
}
/* Playback / layers / camera each read as their own row of the panel. */
.control-group {
display: flex;
flex-wrap: wrap;
gap: 8px 12px;
align-items: center;
}
.control-group + .control-group {
padding-top: 6px;
border-top: 1px solid rgba(255, 255, 255, 0.16);
}
/* Per-asset checkboxes hang off the master "Scene" toggle. */
.control-subgroup {
display: inline-flex;
flex-wrap: wrap;
gap: 8px 10px;
align-items: center;
padding-left: 10px;
border-left: 1px solid rgba(255, 255, 255, 0.16);
}
.control-subgroup:empty {
display: none;
}
.hidden {
display: none;
}

View File

@@ -0,0 +1,672 @@
(function () {
"use strict";
const config = window.OSM_ASSET_PREVIEW_CONFIG || {};
const statusEl = document.getElementById("status");
const diagnosticsEl = document.getElementById("diagnostics");
const loadingOverlay = document.getElementById("loadingOverlay");
const toggleCruise = document.getElementById("toggleCruise");
const toggleFollow = document.getElementById("toggleFollow");
const toggleScene = document.getElementById("toggleScene");
const toggleRoutes = document.getElementById("toggleRoutes");
const toggleVehicles = document.getElementById("toggleVehicles");
const toggleFps = document.getElementById("toggleFps");
const toggleDiagnostics = document.getElementById("toggleDiagnostics");
const assetToggles = document.getElementById("assetToggles");
const vehicleSelect = document.getElementById("vehicleSelect");
const speedControl = document.getElementById("speedControl");
const speedLabel = document.getElementById("speedLabel");
const cameraButtons = Array.from(document.querySelectorAll("[data-camera]"));
// Status carries two kinds of message: the scene summary, which is what the
// panel should read whenever nothing else is going on, and transient notes
// from a control the user just touched. Keep the summary so the transient
// note can be replaced instead of destroying it.
let baseStatus = "";
const PREVIEW_BACKGROUND = "#d9e0e2";
Cesium.Ion.defaultAccessToken = "";
async function main() {
setLoadingMessage("Loading scene", config.areaId || "");
const metadata = await fetchJson(config.metadataName);
const routeData = await fetchOptionalJson(config.routeName);
const placement = scenePlacement(metadata);
const viewer = createViewer();
setLoadingMessage("Loading model", config.glbName || "");
const assets = await loadSceneAssets(viewer, metadata, placement);
const cruise = addVehicleCruises(viewer, routeData, config.vehicleModelName);
const cameras = createCameraPresets(viewer, metadata, placement, cruise);
buildAssetToggles(assets);
bindRuntimeControls(viewer, assets, cruise, cameras);
startDiagnostics(viewer, metadata, assets, cruise, placement);
cameras.overview();
baseStatus = summaryText(metadata, assets, cruise);
setStatus(baseStatus);
setLoadingMessage("Preparing view", "finalizing materials");
await waitForStableFrames(viewer);
document.body.classList.add("scene-ready");
// Handle for the browser console and for headless checks: everything else
// in here is closed over by the IIFE and unreachable from outside.
window.osmPreview = { viewer, metadata, placement, assets, cruise, cameras };
}
async function fetchJson(url) {
const response = await fetch(url);
if (!response.ok) {
throw new Error("Could not load " + url + ": " + response.status);
}
return response.json();
}
// The route file is an extra on top of the scene, not a precondition for it.
// A missing or unreadable route costs the cruise controls, not the preview.
async function fetchOptionalJson(url) {
try {
return await fetchJson(url);
} catch (error) {
console.warn(error);
return null;
}
}
function createViewer() {
const viewer = new Cesium.Viewer("cesiumContainer", {
animation: false,
timeline: false,
baseLayerPicker: false,
geocoder: false,
navigationHelpButton: false,
sceneModePicker: false,
homeButton: true,
fullscreenButton: true,
infoBox: false,
selectionIndicator: false,
baseLayer: false
});
viewer.scene.globe.show = false;
viewer.scene.globe.depthTestAgainstTerrain = false;
viewer.scene.backgroundColor = Cesium.Color.fromCssColorString(PREVIEW_BACKGROUND);
viewer.scene.skyAtmosphere.show = false;
viewer.scene.skyBox.show = false;
viewer.scene.sun.show = false;
viewer.scene.moon.show = false;
viewer.scene.globe.baseColor = Cesium.Color.fromCssColorString(PREVIEW_BACKGROUND);
return viewer;
}
function setLoadingMessage(title, detail) {
if (!loadingOverlay) return;
const titleEl = loadingOverlay.querySelector("strong");
const detailEl = loadingOverlay.querySelector("span");
if (titleEl) titleEl.textContent = title;
if (detailEl) detailEl.textContent = detail || "";
}
function waitForStableFrames(viewer) {
return new Promise((resolve) => {
const started = performance.now();
let frames = 0;
let done = false;
let remove = function () {};
function finish() {
if (done) return;
done = true;
remove();
resolve();
}
remove = viewer.scene.postRender.addEventListener(() => {
frames += 1;
const elapsed = performance.now() - started;
if ((frames >= 14 && elapsed >= 650) || elapsed >= 2800) {
finish();
}
});
window.setTimeout(finish, 3200);
viewer.scene.requestRender();
});
}
function scenePlacement(metadata) {
const anchor = metadata.anchor || {};
const longitude = Number(anchor.longitude || 0);
const latitude = Number(anchor.latitude || 0);
const height = Number(anchor.height || 0);
const heading = Number(metadata.heading_correction_degrees || 0);
const position = Cesium.Cartesian3.fromDegrees(longitude, latitude, height);
const enu = Cesium.Transforms.eastNorthUpToFixedFrame(position);
const correction = Cesium.Matrix3.fromRotationZ(Cesium.Math.toRadians(heading));
return {
longitude,
latitude,
height,
heading,
position,
modelMatrix: Cesium.Matrix4.multiplyByMatrix3(enu, correction, new Cesium.Matrix4())
};
}
function normalizedAssets(metadata) {
const assets = Array.isArray(metadata.assets) && metadata.assets.length
? metadata.assets
: [{
id: "main",
label: "Scene",
type: "model",
url: metadata.asset || config.glbName,
enabled: true
}];
return assets.filter((asset) => (asset.type || "model") === "model" && asset.url);
}
// One broken entry in metadata.assets should not blank the whole preview, so
// failures are collected and surfaced in the diagnostics panel instead.
async function loadSceneAssets(viewer, metadata, placement) {
const loaded = [];
for (const asset of normalizedAssets(metadata)) {
const id = asset.id || "asset-" + loaded.length;
const label = asset.label || asset.id || asset.url;
try {
const model = await Cesium.Model.fromGltfAsync({
url: asset.url,
modelMatrix: placement.modelMatrix,
scale: Number(asset.scale || 1.0)
});
model.show = asset.enabled !== false;
viewer.scene.primitives.add(model);
loaded.push({ id, label, url: asset.url, model, error: null });
} catch (error) {
console.error(error);
loaded.push({ id, label, url: asset.url, model: null, error });
}
}
if (!loaded.some((asset) => asset.model)) {
throw new Error("No scene model could be loaded (" + loaded.length + " declared)");
}
return loaded;
}
function liveAssets(assets) {
return assets.filter((asset) => asset.model);
}
// A single-asset scene keeps the plain "Scene" checkbox; a multi-asset one
// gets a child checkbox per model with "Scene" acting as the master.
function buildAssetToggles(assets) {
const live = liveAssets(assets);
if (live.length < 2) return;
for (const asset of live) {
const label = document.createElement("label");
const input = document.createElement("input");
input.type = "checkbox";
input.checked = asset.model.show;
input.dataset.assetId = asset.id;
input.addEventListener("change", () => {
asset.model.show = input.checked;
syncSceneMaster(assets);
setStatus(asset.label + (input.checked ? " visible" : " hidden"));
});
label.appendChild(input);
label.appendChild(document.createTextNode(" " + asset.label));
assetToggles.appendChild(label);
asset.toggle = input;
}
}
function syncSceneMaster(assets) {
const live = liveAssets(assets);
const shown = live.filter((asset) => asset.model.show).length;
toggleScene.checked = shown > 0;
toggleScene.indeterminate = shown > 0 && shown < live.length;
}
function bindRuntimeControls(viewer, assets, cruise, cameras) {
const hasVehicles = cruise.vehicles.length > 0;
toggleScene.addEventListener("change", () => {
for (const asset of liveAssets(assets)) {
asset.model.show = toggleScene.checked;
if (asset.toggle) asset.toggle.checked = toggleScene.checked;
}
toggleScene.indeterminate = false;
setStatus(toggleScene.checked ? "Scene visible" : "Scene hidden");
});
toggleRoutes.addEventListener("change", () => {
for (const vehicle of cruise.vehicles) vehicle.routeEntity.show = toggleRoutes.checked;
});
toggleVehicles.addEventListener("change", () => {
for (const vehicle of cruise.vehicles) vehicle.entity.show = toggleVehicles.checked;
});
toggleFps.addEventListener("change", () => {
viewer.scene.debugShowFramesPerSecond = toggleFps.checked;
});
toggleDiagnostics.addEventListener("change", () => {
diagnosticsEl.classList.toggle("hidden", !toggleDiagnostics.checked);
});
let stopFollow = function () {};
if (hasVehicles) {
stopFollow = bindCruiseControls(viewer, cruise);
} else {
// Nothing to drive: disable the cruise half of the panel rather than
// leaving controls that silently do nothing.
for (const el of [toggleCruise, toggleFollow, vehicleSelect, speedControl, toggleRoutes, toggleVehicles]) {
el.disabled = true;
}
toggleCruise.textContent = "Play";
}
for (const button of cameraButtons) {
const preset = cameras[button.dataset.camera];
if (!preset || (button.dataset.camera === "route" && !hasVehicles)) {
button.disabled = true;
continue;
}
button.addEventListener("click", () => {
// Chase-follow reclaims the camera on every clock tick, so a preset
// applied underneath it would be overwritten before the next frame.
stopFollow();
preset();
});
}
}
function bindCruiseControls(viewer, cruise) {
// The slider is authored with a static default; the route file decides the
// real cruise speed, so adopt it before the first input event.
speedControl.value = String(Cesium.Math.clamp(
Math.round(cruise.baseSpeed),
Number(speedControl.min),
Number(speedControl.max)
));
viewer.clock.multiplier = Number(speedControl.value) / cruise.baseSpeed;
speedLabel.textContent = speedControl.value + " m/s";
toggleCruise.addEventListener("click", () => {
viewer.clock.shouldAnimate = !viewer.clock.shouldAnimate;
toggleCruise.textContent = viewer.clock.shouldAnimate ? "Pause" : "Play";
});
speedControl.addEventListener("input", () => {
const value = Number(speedControl.value);
viewer.clock.multiplier = value / cruise.baseSpeed;
speedLabel.textContent = value + " m/s";
});
vehicleSelect.addEventListener("change", () => {
cruise.state.selectedIndex = Number(vehicleSelect.value || 0);
setStatus(selectedVehicle(cruise).label);
});
const follow = createChaseFollow(viewer, () => selectedVehicle(cruise).positions);
function stopFollow() {
if (!follow.enabled) return;
follow.stop();
toggleFollow.textContent = "Follow";
}
toggleFollow.addEventListener("click", () => {
if (follow.enabled) {
stopFollow();
setStatus(baseStatus);
} else {
follow.start();
toggleFollow.textContent = "Free";
setStatus("Following " + selectedVehicle(cruise).label);
}
});
return stopFollow;
}
function addVehicleCruises(viewer, routeData, vehicleModelName) {
const segments = ((routeData && routeData.segments) || [])
.filter((segment) => segment.coordinates && segment.coordinates.length >= 2)
.slice(0, 5);
const speed = Number((routeData && routeData.speedMetersPerSecond) || 8);
const start = Cesium.JulianDate.now();
viewer.clock.startTime = start.clone();
viewer.clock.currentTime = start.clone();
viewer.clock.clockRange = Cesium.ClockRange.UNBOUNDED;
viewer.clock.multiplier = 1;
viewer.clock.shouldAnimate = segments.length > 0;
const vehicles = segments.map((segment, index) => {
const vehicle = addCruiseVehicle(viewer, segment, index, start, speed, vehicleModelName);
const option = document.createElement("option");
option.value = String(index);
option.textContent = "#" + (index + 1) + " " + segment.name + " " + Math.round(segment.lengthMeters) + "m";
vehicleSelect.appendChild(option);
return vehicle;
});
return {
vehicles,
baseSpeed: speed,
state: { selectedIndex: 0 }
};
}
function addCruiseVehicle(viewer, segment, index, start, speed, vehicleModelName) {
const route = prepareRoute(segment);
const positions = new Cesium.CallbackProperty((time, result) => {
return routePosition(route, start, time, speed, result);
}, false);
const flat = [];
for (const coord of segment.coordinates) {
flat.push(coord[0], coord[1], 1.05);
}
const routeColor = [
Cesium.Color.CYAN,
Cesium.Color.LIME,
Cesium.Color.YELLOW,
Cesium.Color.ORANGE,
Cesium.Color.DEEPSKYBLUE
][index % 5];
const routeEntity = viewer.entities.add({
name: "Cruise route " + (index + 1),
polyline: {
positions: Cesium.Cartesian3.fromDegreesArrayHeights(flat),
width: 2,
material: routeColor.withAlpha(0.75),
clampToGround: false
}
});
const vehicle = viewer.entities.add({
name: "Cruise vehicle " + (index + 1),
position: positions,
orientation: routeOrientation(route, start, speed, 0.0),
model: {
uri: vehicleModelName,
scale: 1.0,
minimumPixelSize: 24,
maximumScale: 80
}
});
return {
entity: vehicle,
routeEntity,
positions,
route,
segment,
label: "Vehicle #" + (index + 1) + " | route " + segment.id + " | " + Math.round(segment.lengthMeters) + "m"
};
}
function selectedVehicle(cruise) {
return cruise.vehicles[cruise.state.selectedIndex] || cruise.vehicles[0];
}
function prepareRoute(segment) {
const distances = [0.0];
for (let i = 1; i < segment.coordinates.length; i += 1) {
distances.push(distances[i - 1] + distanceMeters(segment.coordinates[i - 1], segment.coordinates[i]));
}
return {
coordinates: segment.coordinates,
distances,
length: Math.max(1.0, distances[distances.length - 1])
};
}
function routePosition(route, start, time, speed, result) {
const seconds = Math.max(0, Cesium.JulianDate.secondsDifference(time, start));
const distance = (seconds * speed) % route.length;
let index = 1;
while (index < route.distances.length - 1 && route.distances[index] < distance) {
index += 1;
}
const prevDist = route.distances[index - 1];
const nextDist = route.distances[index];
const t = nextDist > prevDist ? (distance - prevDist) / (nextDist - prevDist) : 0;
const a = route.coordinates[index - 1];
const b = route.coordinates[index];
const lon = a[0] + (b[0] - a[0]) * t;
const lat = a[1] + (b[1] - a[1]) * t;
return Cesium.Cartesian3.fromDegrees(lon, lat, 1.15, Cesium.Ellipsoid.WGS84, result);
}
function routeOrientation(route, start, speed, yawDegrees) {
const correction = Cesium.Quaternion.fromAxisAngle(
Cesium.Cartesian3.UNIT_Z,
Cesium.Math.toRadians(yawDegrees)
);
const current = new Cesium.Cartesian3();
const ahead = new Cesium.Cartesian3();
const direction = new Cesium.Cartesian3();
const up = new Cesium.Cartesian3();
const east = new Cesium.Cartesian3();
const north = new Cesium.Cartesian3();
return new Cesium.CallbackProperty((time, result) => {
routePosition(route, start, time, speed, current);
const aheadTime = Cesium.JulianDate.addSeconds(time, 0.8, new Cesium.JulianDate());
routePosition(route, start, aheadTime, speed, ahead);
Cesium.Cartesian3.subtract(ahead, current, direction);
if (Cesium.Cartesian3.magnitudeSquared(direction) < 0.0001) {
return result;
}
Cesium.Cartesian3.normalize(direction, direction);
Cesium.Cartesian3.normalize(current, up);
Cesium.Cartesian3.cross(Cesium.Cartesian3.UNIT_Z, up, east);
if (Cesium.Cartesian3.magnitudeSquared(east) < 0.0001) {
Cesium.Cartesian3.clone(Cesium.Cartesian3.UNIT_X, east);
} else {
Cesium.Cartesian3.normalize(east, east);
}
Cesium.Cartesian3.cross(up, east, north);
Cesium.Cartesian3.normalize(north, north);
const eastComponent = Cesium.Cartesian3.dot(direction, east);
const northComponent = Cesium.Cartesian3.dot(direction, north);
const heading = Math.atan2(eastComponent, northComponent);
const base = Cesium.Transforms.headingPitchRollQuaternion(
current,
new Cesium.HeadingPitchRoll(heading, 0.0, 0.0)
);
return Cesium.Quaternion.multiply(base, correction, result || new Cesium.Quaternion());
}, false);
}
function createChaseFollow(viewer, positionsProvider) {
const scratchPosition = new Cesium.Cartesian3();
const scratchPrevious = new Cesium.Cartesian3();
const scratchDirection = new Cesium.Cartesian3();
const scratchEast = new Cesium.Cartesian3();
const scratchNorth = new Cesium.Cartesian3();
const scratchUp = new Cesium.Cartesian3();
const offset = new Cesium.Cartesian3();
const state = { enabled: false, distance: 36.0, height: 18.0 };
function update(clock) {
const time = clock.currentTime;
const positions = positionsProvider();
const position = positions.getValue(time, scratchPosition);
if (!position) return;
const previousTime = Cesium.JulianDate.addSeconds(time, -0.8, new Cesium.JulianDate());
const previous = positions.getValue(previousTime, scratchPrevious);
if (previous) {
Cesium.Cartesian3.subtract(position, previous, scratchDirection);
} else {
Cesium.Cartesian3.clone(Cesium.Cartesian3.UNIT_X, scratchDirection);
}
if (Cesium.Cartesian3.magnitudeSquared(scratchDirection) < 0.0001) {
Cesium.Cartesian3.clone(Cesium.Cartesian3.UNIT_X, scratchDirection);
}
Cesium.Cartesian3.normalize(scratchDirection, scratchDirection);
Cesium.Cartesian3.normalize(position, scratchUp);
Cesium.Cartesian3.cross(Cesium.Cartesian3.UNIT_Z, scratchUp, scratchEast);
if (Cesium.Cartesian3.magnitudeSquared(scratchEast) < 0.0001) {
Cesium.Cartesian3.clone(Cesium.Cartesian3.UNIT_X, scratchEast);
} else {
Cesium.Cartesian3.normalize(scratchEast, scratchEast);
}
Cesium.Cartesian3.cross(scratchUp, scratchEast, scratchNorth);
Cesium.Cartesian3.normalize(scratchNorth, scratchNorth);
const eastComponent = Cesium.Cartesian3.dot(scratchDirection, scratchEast);
const northComponent = Cesium.Cartesian3.dot(scratchDirection, scratchNorth);
const heading = Math.atan2(eastComponent, northComponent);
Cesium.Cartesian3.fromElements(0.0, -state.distance, state.height, offset);
const transform = Cesium.Transforms.headingPitchRollToFixedFrame(
position,
new Cesium.HeadingPitchRoll(heading, 0.0, 0.0)
);
viewer.camera.lookAtTransform(transform, offset);
}
function onWheel(event) {
if (!state.enabled) return;
event.preventDefault();
const zoom = event.deltaY > 0 ? 1.12 : 0.88;
state.distance = Cesium.Math.clamp(state.distance * zoom, 12.0, 160.0);
state.height = Cesium.Math.clamp(state.height * zoom, 6.0, 90.0);
update(viewer.clock);
}
return {
get enabled() {
return state.enabled;
},
start() {
if (state.enabled) return;
state.enabled = true;
viewer.trackedEntity = undefined;
viewer.canvas.addEventListener("wheel", onWheel, { passive: false });
viewer.clock.onTick.addEventListener(update);
update(viewer.clock);
},
stop() {
if (!state.enabled) return;
state.enabled = false;
viewer.clock.onTick.removeEventListener(update);
viewer.canvas.removeEventListener("wheel", onWheel);
viewer.camera.lookAtTransform(Cesium.Matrix4.IDENTITY);
}
};
}
function createCameraPresets(viewer, metadata, placement, cruise) {
const radius = Math.max(220.0, boundsRadiusMeters(metadata.bounds) || 900.0);
const center = placement.position;
function lookAt(range, headingDegrees, pitchDegrees) {
viewer.camera.lookAt(
center,
new Cesium.HeadingPitchRange(
Cesium.Math.toRadians(headingDegrees),
Cesium.Math.toRadians(pitchDegrees),
range
)
);
// lookAt locks the camera into the target's reference frame; release it
// so orbit and pan keep working from the new vantage point.
viewer.camera.lookAtTransform(Cesium.Matrix4.IDENTITY);
}
return {
overview() {
viewer.camera.flyToBoundingSphere(new Cesium.BoundingSphere(center, radius), {
duration: 0.0
});
},
oblique() {
lookAt(radius * 0.92, 135.0, -28.0);
},
detail() {
lookAt(radius * 0.32, 115.0, -18.0);
},
route() {
if (!cruise.vehicles.length) return;
const vehicle = selectedVehicle(cruise);
const coord = vehicle.segment.coordinates[Math.floor(vehicle.segment.coordinates.length / 2)];
viewer.camera.flyTo({
destination: Cesium.Cartesian3.fromDegrees(coord[0], coord[1], 90.0),
orientation: {
heading: Cesium.Math.toRadians(0.0),
pitch: Cesium.Math.toRadians(-62.0),
roll: 0.0
},
duration: 0.0
});
}
};
}
// Camera-dependent readouts have to track the camera, so refresh off the
// render loop rather than a fixed timer, throttled to stay off the hot path.
function startDiagnostics(viewer, metadata, assets, cruise, placement) {
const center = placement.position;
const stats = metadata.scene_stats || {};
const failed = assets.filter((asset) => asset.error);
let lastUpdate = 0;
function render() {
const cartographic = viewer.camera.positionCartographic;
const distance = Cesium.Cartesian3.distance(viewer.camera.positionWC, center);
const lines = [
"Area: " + (config.areaId || "(unknown)"),
"Anchor: " + placement.longitude.toFixed(7) + ", " + placement.latitude.toFixed(7),
"Camera height: " + Math.round(cartographic.height) + " m",
"Camera range: " + Math.round(distance) + " m",
"Assets: " + liveAssets(assets).length + " model(s)",
"Vehicles: " + cruise.vehicles.length,
"Buildings: " + Number(stats.buildings || 0),
"Trees: " + Number(stats.trees || 0),
"Road layer source: " + (metadata.source_geojson ? "osm2streets" : "OSM fallback")
];
if (failed.length) {
lines.push("Failed assets: " + failed.map((asset) => asset.url).join(", "));
}
diagnosticsEl.textContent = lines.join("\n");
}
viewer.scene.postRender.addEventListener(() => {
const now = performance.now();
if (now - lastUpdate < 250) return;
lastUpdate = now;
render();
});
render();
}
function summaryText(metadata, assets, cruise) {
const stats = metadata.scene_stats || {};
return [
config.areaId,
liveAssets(assets).map((asset) => asset.url).join(", "),
cruise.vehicles.length ? "vehicles " + cruise.vehicles.length : "no drivable route",
"buildings " + Number(stats.buildings || 0),
"trees " + Number(stats.trees || 0)
].filter(Boolean).join(" | ");
}
function boundsRadiusMeters(bounds) {
if (!bounds) return null;
const minLon = Number(bounds.min_lon);
const minLat = Number(bounds.min_lat);
const maxLon = Number(bounds.max_lon);
const maxLat = Number(bounds.max_lat);
if (![minLon, minLat, maxLon, maxLat].every(Number.isFinite)) return null;
return Math.max(
80.0,
distanceMeters([minLon, minLat], [maxLon, maxLat]) * 0.58
);
}
function distanceMeters(a, b) {
const radius = 6371008.8;
const lat1 = Cesium.Math.toRadians(a[1]);
const lat2 = Cesium.Math.toRadians(b[1]);
const dLat = Cesium.Math.toRadians(b[1] - a[1]);
const dLon = Cesium.Math.toRadians(b[0] - a[0]);
const sinLat = Math.sin(dLat / 2);
const sinLon = Math.sin(dLon / 2);
const h = sinLat * sinLat + Math.cos(lat1) * Math.cos(lat2) * sinLon * sinLon;
return 2 * radius * Math.asin(Math.min(1, Math.sqrt(h)));
}
function setStatus(message) {
statusEl.textContent = message;
}
main().catch((error) => {
console.error(error);
setStatus("Failed to load Cesium preview: " + error.message);
document.body.classList.add("scene-error");
setLoadingMessage("Failed to load scene", error.message);
});
}());

164
scripts/lib/scene-layers.js Normal file
View File

@@ -0,0 +1,164 @@
"use strict";
// Single source of truth for the osm2streets render layers.
//
// The same nine layers, in the same order, previously appeared four times:
// the merged-scene z_index table, the scene style JSON, the generated QGIS
// project (layer dict + draw_order), and the README's manual rebuild snippet.
// Adding a layer or changing a z-index meant editing all of them in lockstep,
// and a missed copy produces a silently mis-stacked scene downstream in
// Blender/Cesium. Everything now derives from SCENE_LAYERS.
//
// zIndex doubles as draw order: lowest paints first (bottom of the stack).
// outline: null means "no stroke" (QGIS gets a fully transparent outline).
const SCENE_LAYERS = [
{
id: "road_surface",
splitKey: "roadSurface",
zIndex: 10,
title: "road surface",
fill: "#2b2b28",
outline: "#1e1e1c",
outlineWidth: 0.04,
},
{
id: "intersection_surface",
splitKey: "intersectionSurface",
zIndex: 20,
title: "intersection surface",
fill: "#2b2b28",
outline: "#1e1e1c",
outlineWidth: 0.04,
},
{
id: "sidewalks",
splitKey: "sidewalks",
zIndex: 30,
title: "sidewalks",
fill: "#bebeb6",
outline: "#9c9c94",
outlineWidth: 0.025,
},
{
id: "sidewalk_corners",
splitKey: "sidewalkCorners",
zIndex: 40,
title: "sidewalk corners",
fill: "#bebeb6",
outline: "#9c9c94",
outlineWidth: 0.025,
},
{
id: "lane_separators",
splitKey: "laneSeparators",
zIndex: 50,
title: "lane separators",
fill: "#eeeee6",
outline: null,
outlineWidth: 0,
},
{
id: "center_lines",
splitKey: "centerLines",
zIndex: 60,
title: "center lines",
fill: "#f5be2a",
outline: null,
outlineWidth: 0,
},
{
id: "crosswalks",
splitKey: "crosswalks",
zIndex: 70,
title: "crosswalks",
fill: "#fffff6",
outline: null,
outlineWidth: 0,
},
{
id: "vehicle_stop_lines",
splitKey: "vehicleStopLines",
zIndex: 80,
title: "vehicle stop lines",
fill: "#fffff6",
outline: null,
outlineWidth: 0,
},
{
id: "lane_arrows_webscale",
splitKey: "laneArrows",
zIndex: 90,
title: "lane arrows",
fill: "#fffff6",
outline: "#2b2b28",
outlineAlpha: 200,
outlineWidth: 0.015,
},
];
const SCENE_FILE = "osm2streets_scene.geojson";
const SCENE_STYLE_FILE = "osm2streets_scene_style.json";
function layerFile(layer) {
return `${layer.id}.geojson`;
}
// getCollection(layer) -> FeatureCollection, so callers can source layers from
// the in-memory split (build) or from disk (reimport) with the same merge.
function mergeScene(getCollection) {
return {
type: "FeatureCollection",
features: SCENE_LAYERS.flatMap((layer) => {
const collection = getCollection(layer) || {};
return (collection.features || []).map((feature) => ({
...feature,
properties: {
...(feature.properties || {}),
render_layer: layer.id,
z_index: layer.zIndex,
},
}));
}),
};
}
function sceneStyle() {
return {
version: 1,
geometry: "polygon",
sortProperty: "z_index",
layerProperty: "render_layer",
layers: SCENE_LAYERS.map((layer) => ({
id: layer.id,
zIndex: layer.zIndex,
fill: layer.fill,
outline: layer.outline,
outlineWidth: layer.outlineWidth,
})),
};
}
// QGIS symbol properties want "r,g,b,a" strings rather than hex.
function qgisRgba(hex, alpha = 255) {
if (!hex) return "0,0,0,0";
const match = /^#?([0-9a-f]{6})$/i.exec(hex.trim());
if (!match) {
throw new Error(`Expected #rrggbb color, got: ${hex}`);
}
const value = parseInt(match[1], 16);
const r = (value >> 16) & 0xff;
const g = (value >> 8) & 0xff;
const b = value & 0xff;
return `${r},${g},${b},${alpha}`;
}
module.exports = {
SCENE_LAYERS,
SCENE_FILE,
SCENE_STYLE_FILE,
layerFile,
mergeScene,
sceneStyle,
qgisRgba,
};

View File

@@ -0,0 +1,182 @@
#!/usr/bin/env python3
"""Normalize malformed lane-arrow meshes emitted by osm2streets."""
import argparse
import json
import math
from pathlib import Path
from osgeo import ogr
METERS_PER_DEGREE = 111320.0
ogr.UseExceptions()
def cli_args():
parser = argparse.ArgumentParser(
description="Normalize osm2streets triangulated lane-arrow polygons."
)
parser.add_argument("--input", required=True, type=Path)
parser.add_argument("--outline-simplify-meters", required=True, type=float)
return parser.parse_args()
def point_segment_distance(point, start, end, latitude):
meters_lon = METERS_PER_DEGREE * math.cos(math.radians(latitude))
px = (point[0] - start[0]) * meters_lon
py = (point[1] - start[1]) * METERS_PER_DEGREE
bx = (end[0] - start[0]) * meters_lon
by = (end[1] - start[1]) * METERS_PER_DEGREE
length_squared = bx * bx + by * by
if length_squared == 0:
return math.hypot(px, py)
projection = max(0.0, min(1.0, (px * bx + py * by) / length_squared))
return math.hypot(px - projection * bx, py - projection * by)
def simplify_ring(points, tolerance, latitude):
points = list(points)
changed = True
while changed and len(points) > 3:
changed = False
for index, point in enumerate(points):
previous = points[index - 1]
following = points[(index + 1) % len(points)]
if point_segment_distance(point, previous, following, latitude) <= tolerance:
points.pop(index)
changed = True
break
return points
def tail_edge_candidate(points):
best = None
for index in range(len(points)):
start = points[index]
end = points[(index + 1) % len(points)]
previous = points[index - 1]
following = points[(index + 2) % len(points)]
before = [start[0] - previous[0], start[1] - previous[1]]
after = [following[0] - end[0], following[1] - end[1]]
edge = [end[0] - start[0], end[1] - start[1]]
before_length = math.hypot(*before)
after_length = math.hypot(*after)
edge_length = math.hypot(*edge)
if min(before_length, after_length, edge_length) == 0:
continue
alignment = (
before[0] * after[0] + before[1] * after[1]
) / (before_length * after_length)
if before_length <= edge_length or after_length <= edge_length or alignment >= -0.9:
continue
score = -alignment * min(before_length, after_length) / edge_length
if best is None or score > best[0]:
best = (score, index, before, after, before_length, after_length)
return best
def square_arrow_tail(points, latitude):
# A normalized straight arrow has seven exterior vertices. Other arrow
# silhouettes are left untouched because their tail cannot be inferred safely.
if len(points) != 7:
return points
meters_lon = METERS_PER_DEGREE * math.cos(math.radians(latitude))
origin = points[0]
local = [
[
(point[0] - origin[0]) * meters_lon,
(point[1] - origin[1]) * METERS_PER_DEGREE,
]
for point in points
]
candidate = tail_edge_candidate(local)
if candidate is None:
return points
_, index, before, after, before_length, after_length = candidate
axis = [
before[0] / before_length - after[0] / after_length,
before[1] / before_length - after[1] / after_length,
]
axis_length = math.hypot(*axis)
if axis_length == 0:
return points
axis = [axis[0] / axis_length, axis[1] / axis_length]
end_index = (index + 1) % len(local)
midpoint = [
(local[index][0] + local[end_index][0]) / 2.0,
(local[index][1] + local[end_index][1]) / 2.0,
]
for point_index in (index, end_index):
offset = [
local[point_index][0] - midpoint[0],
local[point_index][1] - midpoint[1],
]
projection = offset[0] * axis[0] + offset[1] * axis[1]
local[point_index][0] -= projection * axis[0]
local[point_index][1] -= projection * axis[1]
points[point_index] = [
origin[0] + local[point_index][0] / meters_lon,
origin[1] + local[point_index][1] / METERS_PER_DEGREE,
]
return points
def normalize_polygon(geometry, tolerance):
if geometry.GetGeometryName() == "MULTIPOLYGON":
geometry = geometry.UnionCascaded()
if geometry is None or geometry.GetGeometryName() != "POLYGON":
raise ValueError("triangle merge did not produce a Polygon")
source_ring = geometry.GetGeometryRef(0)
points = [source_ring.GetPoint(i)[:2] for i in range(source_ring.GetPointCount() - 1)]
if len(points) <= 3:
raise ValueError("arrow exterior has too few points")
latitude = sum(point[1] for point in points) / len(points)
points = simplify_ring(points, tolerance, latitude)
points = square_arrow_tail(points, latitude)
normalized = ogr.Geometry(ogr.wkbPolygon)
outer = ogr.Geometry(ogr.wkbLinearRing)
for point in points + [points[0]]:
outer.AddPoint_2D(*point)
normalized.AddGeometry(outer)
for index in range(1, geometry.GetGeometryCount()):
normalized.AddGeometry(geometry.GetGeometryRef(index))
if normalized.IsEmpty() or not normalized.IsValid():
raise ValueError("normalized arrow geometry is invalid")
return normalized
def normalize_file(input_path, tolerance):
with input_path.open("r", encoding="utf-8") as handle:
collection = json.load(handle)
for index, feature in enumerate(collection.get("features", [])):
geometry = ogr.CreateGeometryFromJson(json.dumps(feature.get("geometry", {})))
if geometry is None or geometry.IsEmpty():
raise ValueError(f"feature {index} has no usable geometry")
try:
normalized = normalize_polygon(geometry, tolerance)
except ValueError as error:
raise ValueError(f"feature {index}: {error}") from error
feature["geometry"] = json.loads(
normalized.ExportToJson(options=["COORDINATE_PRECISION=15"])
)
temp_path = input_path.with_name(f".{input_path.name}.tmp")
with temp_path.open("w", encoding="utf-8") as handle:
json.dump(collection, handle, ensure_ascii=False, separators=(",", ":"))
handle.write("\n")
temp_path.replace(input_path)
def main():
args = cli_args()
if args.outline_simplify_meters < 0:
raise ValueError("--outline-simplify-meters must be non-negative")
normalize_file(args.input.resolve(), args.outline_simplify_meters)
if __name__ == "__main__":
main()

270
scripts/parity.js Normal file
View File

@@ -0,0 +1,270 @@
#!/usr/bin/env node
"use strict";
// Parity harness for the osmassets refactor (docs/refactor-plan.md).
//
// node scripts/parity.js capture <label> [--areas a,b] [--stages blender,cesium]
// node scripts/parity.js compare <labelA> <labelB>
//
// capture runs the pipeline and snapshots everything that must not change:
// the stage stdout markers, a structural digest of the .blend, a structural
// digest of the .glb, and a hash of the render PNG. compare diffs two
// snapshots field by field.
//
// Snapshots live under outputs/_refactor-baseline/<label>/, which is inside
// the gitignored outputs/ tree — baselines are local scratch, not artifacts.
const fs = require("fs");
const path = require("path");
const crypto = require("crypto");
const { spawnSync } = require("child_process");
const repoRoot = path.resolve(__dirname, "..");
const baselineRoot = path.join(repoRoot, "outputs", "_refactor-baseline");
const DEFAULT_AREAS = ["nantaizi-lake-innovation-valley", "hanyang-block"];
// Fields a control run (identical code, run twice) proved unstable. They are
// still recorded — a human reading a snapshot wants them — but comparing them
// would bury real regressions under noise.
//
// files.*.sha256 / .bytes for blend, glb, render
// .blend embeds absolute paths and packs images in hash-map order, so the
// file hash moves while the structural digest stays put. The EEVEE render
// is likewise not bit-reproducible.
// glbDigest.fileBytes / .buffers / .counts.accessors
// The glTF exporter deduplicates identical accessors. smart_project UVs
// carry float noise, so two runs can differ by one shared UV accessor
// (observed: 399 vs 398 accessors, 720 bytes) with identical nodes,
// meshes, primitives, materials and images.
//
// What remains compared is the real contract: the SCENE_DONE / CESIUM markers,
// the full .blend structural digest (objects, meshes, materials, custom
// properties), and the GLB node/mesh/material/image structure.
const IGNORED_PATHS = new Set([
"capturedAt",
"durationMs",
"label",
"files.blend.sha256",
"files.glb.sha256",
"files.glb.bytes",
"files.render.sha256",
"files.render.bytes",
"glbDigest.fileBytes",
"glbDigest.buffers",
"glbDigest.counts.accessors",
]);
function main() {
const [command, ...rest] = process.argv.slice(2);
if (command === "capture") return capture(rest);
if (command === "compare") return compare(rest);
console.error("usage: parity.js capture <label> [--areas a,b] [--stages s]");
console.error(" parity.js compare <labelA> <labelB>");
process.exit(1);
}
function parseFlags(argv) {
const flags = {};
const positional = [];
for (let i = 0; i < argv.length; i += 1) {
if (argv[i].startsWith("--") && i + 1 < argv.length) {
flags[argv[i].slice(2)] = argv[i + 1];
i += 1;
} else {
positional.push(argv[i]);
}
}
return { flags, positional };
}
function capture(argv) {
const { flags, positional } = parseFlags(argv);
const label = positional[0];
if (!label) throw new Error("capture needs a label");
const areas = (flags.areas ? flags.areas.split(",") : DEFAULT_AREAS)
.map((a) => a.trim())
.filter(Boolean);
const stages = flags.stages || "blender,cesium";
for (const area of areas) {
const configPath = path.join(repoRoot, "config", "areas", `${area}.json`);
if (!fs.existsSync(configPath)) throw new Error(`No config for area: ${area}`);
const config = JSON.parse(fs.readFileSync(configPath, "utf8"));
const areaDir = path.join(repoRoot, "outputs", area);
const stem = area;
const outDir = path.join(baselineRoot, label, area);
fs.mkdirSync(outDir, { recursive: true });
console.log(`\n=== parity capture [${label}] ${area} (stages: ${stages}) ===`);
const started = Date.now();
const run = spawnSync(process.execPath, [
path.join(repoRoot, "scripts", "build-area.js"),
"--config", configPath,
"--stages", stages,
], { cwd: repoRoot, encoding: "utf8", maxBuffer: 64 * 1024 * 1024 });
const stdout = `${run.stdout || ""}`;
const stderr = `${run.stderr || ""}`;
process.stdout.write(stdout);
if (run.status !== 0) {
process.stderr.write(stderr);
throw new Error(`build-area failed for ${area} (exit ${run.status})`);
}
const snapshot = {
label,
area,
stages,
capturedAt: new Date().toISOString(),
durationMs: Date.now() - started,
markers: {
scene: parseMarker(stdout, "SCENE_DONE"),
cesium: parseMarker(stdout, "CESIUM_EXPORT_DONE"),
},
files: {},
};
const blend = path.join(areaDir, `${stem}.blend`);
if (fs.existsSync(blend)) {
snapshot.files.blend = fileStat(blend);
snapshot.blendDigest = blendDigest(config, blend, path.join(outDir, "blend-digest.json"));
}
const glb = path.join(areaDir, `${stem}.glb`);
if (fs.existsSync(glb)) {
snapshot.files.glb = fileStat(glb);
snapshot.glbDigest = glbDigest(glb, path.join(outDir, "glb-digest.json"));
}
for (const [key, file] of [
["render", path.join(areaDir, `${stem}.png`)],
["metadata", path.join(areaDir, `${stem}.json`)],
]) {
if (fs.existsSync(file)) snapshot.files[key] = fileStat(file);
}
if (fs.existsSync(path.join(areaDir, `${stem}.json`))) {
snapshot.metadata = JSON.parse(
fs.readFileSync(path.join(areaDir, `${stem}.json`), "utf8"),
);
}
writeJson(path.join(outDir, "snapshot.json"), snapshot);
console.log(`Snapshot: ${path.join(outDir, "snapshot.json")}`);
}
}
function parseMarker(stdout, marker) {
const line = stdout.split("\n").find((l) => l.startsWith(`${marker} `));
if (!line) return null;
try {
return JSON.parse(line.slice(marker.length + 1));
} catch (error) {
return { unparsed: line };
}
}
function fileStat(file) {
const buffer = fs.readFileSync(file);
return {
bytes: buffer.length,
sha256: crypto.createHash("sha256").update(buffer).digest("hex"),
};
}
function blendDigest(config, blend, outFile) {
const blenderApp = config.blenderApp || "/Applications/Blender.app";
const blender = path.join(blenderApp, "Contents", "MacOS", "Blender");
const run = spawnSync(blender, [
"--background", "--factory-startup",
"--python", path.join(repoRoot, "blender", "tools", "scene_digest.py"),
"--", "--blend", blend, "--out", outFile,
], { cwd: repoRoot, encoding: "utf8", maxBuffer: 64 * 1024 * 1024 });
if (run.status !== 0) {
process.stderr.write(`${run.stdout || ""}${run.stderr || ""}`);
throw new Error(`scene_digest failed for ${blend}`);
}
return JSON.parse(fs.readFileSync(outFile, "utf8"));
}
function glbDigest(glb, outFile) {
const run = spawnSync(process.execPath, [
path.join(repoRoot, "scripts", "glb-digest.js"), glb, "--out", outFile,
], { cwd: repoRoot, encoding: "utf8", maxBuffer: 64 * 1024 * 1024 });
if (run.status !== 0) {
process.stderr.write(`${run.stdout || ""}${run.stderr || ""}`);
throw new Error(`glb-digest failed for ${glb}`);
}
return JSON.parse(fs.readFileSync(outFile, "utf8"));
}
function writeJson(file, value) {
fs.mkdirSync(path.dirname(file), { recursive: true });
fs.writeFileSync(file, `${JSON.stringify(value, null, 2)}\n`);
}
function compare(argv) {
const [a, b] = argv;
if (!a || !b) throw new Error("compare needs two labels");
const areas = fs.readdirSync(path.join(baselineRoot, a))
.filter((entry) => fs.existsSync(path.join(baselineRoot, a, entry, "snapshot.json")));
let differences = 0;
for (const area of areas) {
const left = readSnapshot(a, area);
const right = readSnapshot(b, area);
if (!right) {
console.log(`\n[${area}] missing in ${b} — skipped`);
continue;
}
const diffs = [];
diffValues("", left, right, diffs);
console.log(`\n=== ${area}: ${a} vs ${b} ===`);
if (!diffs.length) {
console.log("identical");
} else {
differences += diffs.length;
for (const line of diffs.slice(0, 200)) console.log(line);
if (diffs.length > 200) console.log(`${diffs.length - 200} more`);
}
}
console.log(`\n${differences === 0 ? "PARITY OK" : `PARITY DIFF (${differences})`}`);
process.exitCode = differences === 0 ? 0 : 2;
}
function readSnapshot(label, area) {
const file = path.join(baselineRoot, label, area, "snapshot.json");
return fs.existsSync(file) ? JSON.parse(fs.readFileSync(file, "utf8")) : null;
}
function diffValues(pathKey, left, right, out) {
if (IGNORED_PATHS.has(pathKey)) return;
if (left === right) return;
const bothObjects = left && right && typeof left === "object" && typeof right === "object";
if (!bothObjects) {
out.push(` ${pathKey || "<root>"}: ${format(left)} -> ${format(right)}`);
return;
}
if (Array.isArray(left) !== Array.isArray(right)) {
out.push(` ${pathKey}: array/object mismatch`);
return;
}
if (Array.isArray(left)) {
if (left.length !== right.length) {
out.push(` ${pathKey}.length: ${left.length} -> ${right.length}`);
}
const limit = Math.min(left.length, right.length);
for (let i = 0; i < limit; i += 1) {
diffValues(`${pathKey}[${i}]`, left[i], right[i], out);
}
return;
}
const keys = new Set([...Object.keys(left), ...Object.keys(right)]);
for (const key of [...keys].sort()) {
diffValues(pathKey ? `${pathKey}.${key}` : key, left[key], right[key], out);
}
}
function format(value) {
if (value === undefined) return "<missing>";
const text = JSON.stringify(value);
return text && text.length > 120 ? `${text.slice(0, 117)}` : text;
}
main();

179
scripts/reimport-gpkg.js Executable file
View File

@@ -0,0 +1,179 @@
#!/usr/bin/env node
"use strict";
// Reverse of the intermediates stage: pull manually-edited layers back out of
// <area>.gpkg into osm2streets_web_out/*.geojson and rebuild the merged scene.
//
// Use after hand-fixing geometry in QGIS. Re-running intermediates would
// regenerate the GeoPackage from OSM and throw those edits away.
//
// Every layer is exported to a staging directory and parsed before anything in
// outDir is touched: ogr2ogr exits non-zero on a missing layer but still leaves
// a zero-byte file behind, so a partial export must not reach the output tree.
const fs = require("fs");
const path = require("path");
const os = require("os");
const { execFileSync } = require("child_process");
const {
SCENE_LAYERS,
SCENE_FILE,
SCENE_STYLE_FILE,
layerFile,
mergeScene,
sceneStyle,
} = require("./lib/scene-layers");
const args = parseArgs(process.argv.slice(2));
const config = loadConfig(args);
const qgisApp = config.qgisApp || "/Applications/QGIS.app";
const qgisMacOS = path.join(qgisApp, "Contents", "MacOS");
const ogr2ogr = path.join(qgisMacOS, "ogr2ogr");
const ogrinfo = path.join(qgisMacOS, "ogrinfo");
const outDir = path.resolve(requireText(config.outDir, "outDir"));
const gpkgPath = path.resolve(requireText(config.gpkg, "gpkg"));
for (const exe of [ogr2ogr, ogrinfo]) {
if (!fs.existsSync(exe)) {
throw new Error(`QGIS executable not found: ${exe}`);
}
}
if (!fs.existsSync(gpkgPath)) {
throw new Error(`GeoPackage not found: ${gpkgPath}\nRun the intermediates stage first.`);
}
if (!fs.existsSync(outDir)) {
throw new Error(`GeoJSON output directory not found: ${outDir}`);
}
console.log(`Reimport: ${gpkgPath}`);
console.log(`Target: ${outDir}`);
const present = gpkgLayers();
const missing = SCENE_LAYERS.filter((layer) => !present.has(layer.id)).map((layer) => layer.id);
if (missing.length) {
throw new Error(
`GeoPackage is missing ${missing.length} layer(s): ${missing.join(", ")}\n` +
`Present: ${[...present].join(", ") || "(none)"}`,
);
}
const stagingDir = fs.mkdtempSync(path.join(os.tmpdir(), "osm2streets-reimport-"));
try {
const staged = SCENE_LAYERS.map((layer) => {
const stagedPath = path.join(stagingDir, layerFile(layer));
exportLayer(layer.id, stagedPath);
const collection = readCollection(stagedPath, layer.id);
console.log(`${layer.id}\tfeatures=${collection.features.length}`);
return { layer, stagedPath, collection };
});
for (const item of staged) {
// Copy rather than rename: the staging dir may be on another filesystem.
fs.copyFileSync(item.stagedPath, path.join(outDir, layerFile(item.layer)));
}
const byId = new Map(staged.map((item) => [item.layer.id, item.collection]));
const scene = mergeScene((layer) => byId.get(layer.id));
fs.writeFileSync(path.join(outDir, SCENE_FILE), JSON.stringify(scene));
fs.writeFileSync(
path.join(outDir, SCENE_STYLE_FILE),
JSON.stringify(sceneStyle(), null, 2),
);
console.log(`${path.join(outDir, SCENE_FILE)}\tfeatures=${scene.features.length}`);
const empty = staged.filter((item) => item.collection.features.length === 0);
if (empty.length) {
console.warn(`Warning: empty layer(s): ${empty.map((item) => item.layer.id).join(", ")}`);
}
} finally {
fs.rmSync(stagingDir, { recursive: true, force: true });
}
function parseArgs(argv) {
const out = {};
for (let i = 0; i < argv.length; i += 1) {
const arg = argv[i];
if (!arg.startsWith("--")) continue;
const key = arg.slice(2).replace(/-([a-z])/g, (_, c) => c.toUpperCase());
const next = argv[i + 1];
if (!next || next.startsWith("--")) {
out[key] = "true";
} else {
out[key] = next;
i += 1;
}
}
return out;
}
function loadConfig(cliArgs) {
const base = {};
if (cliArgs.config) {
const file = path.resolve(cliArgs.config);
if (!fs.existsSync(file)) {
throw new Error(`Config file not found: ${file}`);
}
Object.assign(base, JSON.parse(fs.readFileSync(file, "utf8")));
}
for (const key of ["qgisApp", "outDir", "gpkg"]) {
if (cliArgs[key] !== undefined) base[key] = cliArgs[key];
}
return base;
}
function requireText(value, key) {
if (typeof value !== "string" || value.trim() === "") {
throw new Error(`Missing config key: ${key}`);
}
return value;
}
function gdalEnv() {
return {
PROJ_LIB: path.join(qgisApp, "Contents", "Resources", "qgis", "proj"),
GDAL_DATA: path.join(qgisApp, "Contents", "Resources", "qgis", "gdal"),
};
}
function gpkgLayers() {
const output = execFileSync(ogrinfo, ["-q", gpkgPath], {
encoding: "utf8",
env: { ...process.env, ...gdalEnv() },
});
const names = new Set();
for (const line of output.split("\n")) {
const match = /^\s*\d+:\s+(\S+)/.exec(line);
if (match) names.add(match[1]);
}
return names;
}
function exportLayer(layerName, destination) {
// No COORDINATE_PRECISION here on purpose: the default already round-trips
// full double precision, and setting it explicitly makes GDAL run its
// precision-reduction pass, which drops vertices that collapse at the given
// resolution (measured: 28 points lost across 7 lane-arrow polygons).
execFileSync(ogr2ogr, [
"-f", "GeoJSON",
destination,
gpkgPath,
layerName,
], {
stdio: "inherit",
env: { ...process.env, ...gdalEnv() },
});
}
function readCollection(file, layerName) {
let parsed;
try {
parsed = JSON.parse(fs.readFileSync(file, "utf8"));
} catch (error) {
throw new Error(`Layer '${layerName}' did not export valid GeoJSON: ${error.message}`);
}
if (parsed.type !== "FeatureCollection" || !Array.isArray(parsed.features)) {
throw new Error(`Layer '${layerName}' did not export a FeatureCollection`);
}
return parsed;
}