kawagiri/TH1
1
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 9 Wiki Activity
TH1/.codex/skills/th1-server-backend/references/server-map.md

20 KiB
Raw Blame History

TH1 Server Backend Map

Scope

This reference covers the TH1 backend upload service and its local data/tooling:

  • Backend: Tools/OSS/game-upload-function/
  • Architecture doc: MD/GameMDFramework/15-服务端-GameUploadFunction.md
  • Unity runtime client: Unity/Assets/Scripts/TH1_Logic/Oss/
  • Unity editor OSS tools: Unity/Assets/Scripts/TH1_Logic/Editor/OssEditorWindow.cs and OssDownloadService.cs
  • Pulled OSS collect data: Tools/OSS/Data/
  • Player bug report viewer/restorer: Tools/PlayerBugViewer/
  • Player multilingual report viewer: Tools/PlayerMultilingualReportViewer/
  • Player questionnaire answer viewer: Tools/PlayerQuestionnaireViewer/

Function Compute Service

Tools/OSS/game-upload-function/index.js is a Node.js CommonJS HTTP server intended for Aliyun Function Compute.

Key constants:

  • PORT = 9000
  • MAX_BODY_SIZE = 1024
  • TOKEN_CACHE_DURATION_MS = 5 * 60 * 1000
  • STEAM_AUTH_CACHE_DURATION_MS = 10 * 60 * 1000
  • MAX_STANDARD_UPLOAD_SIZE = 3 * 1024 * 1024
  • MAX_BUG_REPORT_UPLOAD_SIZE = 10 * 1024 * 1024
  • MAX_MULTILINGUAL_REPORT_UPLOAD_SIZE = 1 * 1024 * 1024
  • MAX_QUESTIONNAIRE_UPLOAD_SIZE = 512 * 1024
  • STS_DURATION_SECONDS = 900

Upload types are centralized in UPLOAD_TYPE_CONFIG:

  • ossdata: cached STS credential, .dat, version root prefix, 3 MB limit.
  • collectdata: cached STS credential, .dat, collect/{version} prefix, 3 MB limit.
  • bugreport: uncached STS credential, .zip, bugreport/{version} prefix, 10 MB limit.
  • multilingualreport: uncached STS credential, .zip, multilingualreport/{version} prefix, 1 MB limit.
  • questionnaire: uncached STS credential, .json, questionnaire/{version} prefix, 512 KB limit.

Public Unity function URL currently appears in Unity as:

https://get-sts-token-qltjykaafr.cn-shanghai.fcapp.run

Dependencies:

  • @alicloud/pop-core
  • tablestore
  • Node engine: >=18.0.0

Required environment variables:

  • ACCESS_KEY_ID
  • ACCESS_KEY_SECRET
  • ROLE_ARN
  • BUCKET_NAME
  • STEAM_API_KEY
  • STEAM_APP_ID
  • OTS_ENDPOINT
  • OTS_INSTANCE

Never place those values in repository files.

HTTP Contract

Request:

{
  "action": "",
  "steamId": "76561198...",
  "authTicket": "hex steam ticket",
  "version": "0.7.0",
  "type": "ossdata"
}

Rules:

  • Method must be POST; OPTIONS is accepted for CORS.
  • Body must be JSON and no larger than 1024 bytes.
  • steamId is required.
  • authTicket is required when action=steamauth is not already covered by a valid Steam identity cache, or when an upload misses the Steam identity cache.
  • version is optional; empty or missing is treated as null then path segment common.
  • action is optional; exact steamauth runs Steam pre-verification only. Other values default to upload flow.
  • type is optional; valid values are ossdata, collectdata, bugreport, multilingualreport, and questionnaire. Unknown values default to ossdata.

Successful response maps directly to Unity StsCredentials:

{
  "accessKeyId": "STS...",
  "accessKeySecret": "...",
  "securityToken": "...",
  "endpoint": "oss-cn-shanghai.aliyuncs.com",
  "bucket": "th1-oss",
  "objectKey": "0.7.0/76561198.../1776000000000.dat",
  "policy": "base64",
  "signature": "base64-hmac-sha1",
  "expiresIn": 900
}

Steam pre-verification response:

{
  "success": true,
  "cached": false,
  "steamId": "76561198...",
  "version": "0.7.0",
  "expiresIn": 600
}

Expected status codes:

  • 400: missing steamId, or missing authTicket after Steam identity cache miss, malformed/invalid request handling.
  • 403: Steam ticket verification failed.
  • 405: non-POST request.
  • 413: request body exceeds 1024 bytes.
  • 500: missing env vars, STS failure, Tablestore/client exception, or uncaught server exception.

Runtime Flow

  1. CORS and method checks.
  2. Required environment variable checks.
  3. Parse JSON and normalize version/action.
  4. If action=steamauth, verify via Steam Web API unless Steam identity cache already exists, then write/return Steam identity cache status.
  5. Upload flow normalizes type and checks STS credential cache for cacheable types.
  6. If STS cache misses, check Steam identity cache first.
  7. If Steam identity cache misses, call Steam Web API AuthenticateUserTicket/v1.
  8. If Steam identity is valid, call Aliyun STS AssumeRole.
  9. Build exact objectKey.
  10. Generate Post Policy and HMAC-SHA1 signature.
  11. Save STS credential to Tablestore only for cacheable upload types.
  12. Return credential response to Unity.

Steam verification retries 3 times and uses a 15-second HTTPS timeout.

OSS Paths

Backend-generated object keys:

  • ossdata with version: {version}/{steamId}/{timestamp}.dat
  • ossdata without version: common/{steamId}/{timestamp}.dat
  • collectdata with version: collect/{version}/{steamId}/{timestamp}.dat
  • collectdata without version: collect/common/{steamId}/{timestamp}.dat
  • bugreport with version: bugreport/{version}/{steamId}/{timestamp}-{random}.zip
  • bugreport without version: bugreport/common/{steamId}/{timestamp}-{random}.zip
  • multilingualreport with version: multilingualreport/{version}/{steamId}/{timestamp}-{random}.zip
  • multilingualreport without version: multilingualreport/common/{steamId}/{timestamp}-{random}.zip
  • questionnaire with version: questionnaire/{version}/{steamId}/{timestamp}-{random}.json
  • questionnaire without version: questionnaire/common/{steamId}/{timestamp}-{random}.json

Bug report package shape:

  • manifest.json
  • description.txt
  • saves/{single|multi}/map_archive_begin[_multi]_{mapId}.dat
  • saves/{single|multi}/map_archive_continue[_multi]_{mapId}.dat or map_archive_end[_multi]_{mapId}.dat

Bug report manifest includes:

  • Identity/time: reportId, createdAtUtc, createdAtLocal, timezone, steamId.
  • Version/runtime: version, unityVersion, platform.
  • CrashSight correlation: crashSightDeviceId. CrashSightManager.Initialize sets the same value on CrashSight via CrashSightAgent.SetDeviceId.
  • Device ID source: CrashSightManager.GetCrashSightDeviceId() prefers SystemInfo.deviceUniqueIdentifier; if Unity returns an empty/unsupported identifier, it generates and persists a th1-{guid} fallback in PlayerPrefs. If even PlayerPrefs is unavailable, it uses a process-local th1-device-id-unavailable-{guid} fallback. Treat it as a stable correlation ID, not a cryptographic global uniqueness guarantee.
  • Device snapshot: deviceName, deviceModel, operatingSystem, processorType, processorCount, systemMemorySizeMb, graphicsDeviceName, graphicsMemorySizeMb.
  • Content/archive: description, archiveCount, archives[].

Questionnaire payload shape:

  • Plain UTF-8 JSON, not zip.
  • Schema: th1.questionnaire-answer.v1.
  • Identity/time: responseId, questionnaireId, submittedAtUnix, submittedAtUtc, createdAtUtc, createdAtLocal, timezone, steamId.
  • Version/runtime/device fields mirror player bug reports where useful: version, unityVersion, platform, crashSightDeviceId, device and graphics snapshot.
  • Answer content lives under answerSheet and contains the local QuestionnaireAnswerSheet object with Answers[].

Local collect download mapping:

  • OSS collect/0.7.0/{steamId}/{ts}.dat
  • Local Tools/OSS/Data/0.7.0/{steamId}/{ts}.dat

JSON export mapping:

  • Source .dat: Tools/OSS/Data/{version}/{steamId}/{ts}.dat
  • Export JSON: Tools/OSS/Data/JsonExport/{version}/{steamId}/{ts}.json

Current local data layout has version folders such as 0.6.10 and 0.7.0, plus JsonExport.

Tablestore Cache

Table: Players

STS credential primary key:

  • PlayerId = {steamId}#{type}

Attributes:

  • accessKeyId
  • accessKeySecret
  • securityToken
  • endpoint
  • bucket
  • objectKey
  • policy
  • signature
  • issuedAt
  • stsExpireAt
  • version

Cache hit requires all of:

  • version matches after empty values normalize to no-version.
  • Date.now() - issuedAt <= 5 minutes.
  • stsExpireAt - Date.now() >= 2 minutes.

The cache key includes upload type, so map upload and collect upload tokens do not collide.

bugreport, multilingualreport, and questionnaire deliberately bypass the STS credential cache because each submission must get a unique object key.

Steam identity primary key:

  • PlayerId = {steamId}#steamauth

Steam identity attributes:

  • verifiedAt
  • authExpireAt
  • version

Steam identity cache hit requires:

  • authExpireAt > Date.now()

This cache does not authorize a broad write by itself. It only allows subsequent upload credential requests to skip the slow Steam Web API call while still receiving a fresh, exact-key STS/Post Policy.

Security Invariants

Keep these properties unless the user explicitly accepts the risk:

  • Steam ticket must be checked on cache miss.
  • Steam response params.steamid must equal requested steamId.
  • STS role session name should remain short enough for Aliyun constraints.
  • STS inline policy should allow only oss:PutObject.
  • STS resource should be exactly acs:oss:*:*:{bucket}/{objectKey}.
  • Post Policy should include exact bucket, exact key, and content-length-range.
  • Unity should not upload files larger than its own guard limit unless both client and backend policy are changed together.
  • Steam identity cache duration should remain short; current value is 10 minutes.
  • bugreport should remain capped at 10 MB and should not reuse cached object keys.
  • questionnaire should remain capped at 512 KB and should not reuse cached object keys.

Unity Client Contract

Runtime files:

  • OssManager.cs: builds OssData/CollectData, obtains Steam ticket, caches separate STS credentials for ossdata and collectdata, and starts async uploads.
  • StsTokenService.cs: sends JSON to Function Compute and parses StsCredentials or SteamAuthWarmupResponse.
  • OssUploadService.cs: builds multipart PostObject requests to https://{bucket}.{endpoint}.
  • PlayerBugReportService.cs: builds player bug report zip packages and selects recent start + continue/end save pairs.
  • QuestionnaireUploadService.cs: builds questionnaire answer JSON payloads using schema th1.questionnaire-answer.v1.
  • OssData.cs: MemoryPack payload containing StartMap, Actions, and CollectData.

Important Unity details:

  • StsTokenService always sends the current client version from ConfigManager.Instance.VersionCfg.CurVersionInfo.Version.
  • Request timeout is 30 seconds for STS and 60 seconds for OSS upload.
  • Main.Update calls OssManager.Instance.UpdateSteamAuthWarmup() so Steam identity is pre-verified while the player is online.
  • Runtime standard upload guard rejects payloads larger than 3 MB; bugreport upload guard rejects packages larger than 10 MB; questionnaire upload guard rejects packages larger than 512 KB.
  • Multipart fields include key, OSSAccessKeyId, policy, Signature, x-oss-security-token, then file last.
  • OssManager.UploadMapData and UploadCollectData skip multiplayer uploads on non-lobby-owner clients.
  • OssManager.UploadPlayerBugReportAsync requests type=bugreport and uploads application/zip.
  • OssManager.UploadPlayerMultilingualReportAsync requests type=multilingualreport and uploads application/zip.
  • OssManager.UploadQuestionnaireAnswerAsync requests type=questionnaire and uploads application/json.

Player Bug Report Tooling

Unity temporary submit UI:

  • Unity/Assets/Scripts/TH1_Logic/Editor/PlayerBugReportEditorWindow.cs
  • Menu: Tools/玩家 Bug 汇报
  • Inputs: version, player description, include recent archive, single-player archive, multiplayer archive.
  • Default archive selection picks the most recent valid session.

Save session pairing:

  • Start file: map_archive_begin[_multi]_{mapId}.dat
  • Companion file: latest map_archive_continue[_multi]_{mapId}.dat or map_archive_end[_multi]_{mapId}.dat
  • Both .dat and .dat.bak candidates may be scanned, but zip entries preserve the source filename.

Standalone developer viewer:

  • Folder: Tools/PlayerBugViewer/
  • Entry point: 启动玩家Bug查看器.bat
  • Main script: player_bug_viewer.py
  • Local credentials: config.local.json, ignored by git.
  • Download cache: Data/, ignored by git.

Viewer capabilities:

  • Pull bugreport/ zip objects from OSS using OSS REST V1 HMAC-SHA1 signing.
  • Filter by version and SteamID.
  • Preview manifest fields, attached save files, and player description.
  • Replace local saves by deleting map_archive_*.dat / map_archive_*.dat.bak in the configured local Config directory, then copying archive files from the selected report.

Player Multilingual Report Tooling

Unity temporary submit UI:

  • Unity/Assets/Scripts/TH1_Logic/Editor/PlayerBugReportEditorWindow.cs
  • Menu: Tools/玩家多语言汇报
  • Inputs: version, multilingual ID, selected text, language, and player description.

Standalone developer viewer:

  • Folder: Tools/PlayerMultilingualReportViewer/
  • Entry point: 启动玩家多语言汇报查看器.bat
  • Main script: player_multilingual_report_viewer.py
  • Local credentials: config.local.json, ignored by git.
  • Download cache: Data/, ignored by git.

Viewer capabilities:

  • Pull multilingualreport/ zip objects from OSS using OSS REST V1 HMAC-SHA1 signing.
  • Filter by version, SteamID, language, and multilingual ID.
  • Preview manifest fields, reported text, resolved text, and player description.

Player Questionnaire Tooling

Unity client upload payload:

  • Unity/Assets/Scripts/TH1_Logic/Questionnaire/QuestionnaireUploadService.cs
  • Schema: th1.questionnaire-answer.v1
  • Upload type: questionnaire
  • Content type: application/json
  • Max size: 512 KB
  • Expected object key: questionnaire/{version}/{steamId}/{timestamp}-{random}.json

Standalone developer viewer:

  • Folder: Tools/PlayerQuestionnaireViewer/
  • Entry point: 启动玩家问卷查看器.bat
  • Main script: player_questionnaire_viewer.py
  • Local credentials: config.local.json, ignored by git.
  • Download cache: Data/, ignored by git.

Viewer capabilities:

  • Pull questionnaire/ JSON objects from OSS using OSS REST V1 HMAC-SHA1 signing.
  • Filter by version, questionnaire ID, SteamID, and question ID.
  • Preview response metadata, device fields, and answer details under answerSheet.Answers[].

Unity Editor OSS Tooling

Editor files:

  • OssEditorWindow.cs
  • OssDownloadService.cs
  • SteamEditorWindow.cs
  • OssStatisticEditorWindow.cs is deprecated and retained only for Unity .meta stability.

Menu items:

  • Tools/Oss 编辑器
  • Tools/Steam 上传流程测试器
  • Tools/OSS 导出 JSON (Dashboard)

Editor capabilities:

  • Download all OSS objects under collect/.
  • Save credentials to local EditorPrefs, not version control.
  • Skip local files that already exist.
  • Deserialize collect .dat files with MemoryPack for statistics.
  • Export collect data JSON for the dashboard under Tools/OSS/Data/JsonExport.
  • Test STS request, upload success, wrong object key rejection, and oversized upload rejection.
  • Run the Steam-authenticated end-to-end upload sequence 1-5: action=steamauth, type=ossdata, type=bugreport, type=multilingualreport, and type=questionnaire.

Tools/Steam 上传流程测试器 requires Play Mode with Steam initialized and a valid AuthTicket. Its questionnaire test constructs a small th1.questionnaire-answer.v1 JSON payload, requests type=questionnaire, verifies the returned object key is under questionnaire/ and ends in .json, then uploads with application/json and the 512 KB guard. This tester does not cover collectdata; keep using Tools/Oss 编辑器 for collect upload policy checks.

OssDownloadService uses OSS REST V1 HMAC-SHA1 signing. Its ListObjectsAsync handles continuation tokens and DownloadObjectAsync encodes each object-key path segment while preserving slashes.

Production Triage Notes

Common CrashSight/UnityLogError examples around this service:

  • CollectData upload failed: STS request failed: HTTP/1.1 403 Forbidden
  • Steam verification failed: Ticket for other app
  • STS request failed: Request timeout
  • Cannot resolve destination host
  • Unable to complete SSL connection
  • OSS PostObject 上传失败

Interpretation:

  • Ticket for other app: check STEAM_APP_ID, client build, and whether the ticket was generated for the expected app.
  • Invalid parameter: check malformed/empty ticket, request JSON, and body-size truncation.
  • DNS/SSL/connect/timeout errors: often client network or regional connectivity; do not classify as a server crash without corroborating Function Compute logs.
  • OSS upload failure after STS success: inspect PostObject field names, exact key match, policy expiry, token expiry, endpoint, bucket, and payload size.
  • Cache issues: stale version/type/objectKey almost always mean cache normalization changed on only one side of read/write.
  • Slow upload credential requests after Steam is ready: verify the client is calling action=steamauth, Tablestore has {steamId}#steamauth, and authExpireAt is still in the future.
  • Player bug report upload overwrites: bugreport should bypass STS cache and object keys should include a random suffix.
  • Multilingual report upload overwrites: multilingualreport should bypass STS cache and object keys should include a random suffix.
  • Questionnaire upload failure after STS success: verify JSON is under 512 KB, content type is application/json, object key is questionnaire/*.json, and service/client are both using type=questionnaire.
  • Player bug viewer restore problems: inspect manifest.json archive zipEntry/sourceFileName fields and the configured local Config path.

Deployment And Validation

Local code checks:

  • In Tools/OSS/game-upload-function, run npm install only when dependencies are missing or changed.
  • In Tools/OSS/game-upload-function, run npm run check for the static upload contract check. It includes node -c index.js and verifies upload-type/path/limit policy snippets, including questionnaire.
  • There is no meaningful npm test currently; package.json has the default failing test stub.
  • A local server boot requires all required env vars; without them, HTTP requests should return a controlled 500 listing missing variables.
  • Backend syntax/check command: npm run check from Tools/OSS/game-upload-function.
  • Player bug viewer syntax check: python -m py_compile Tools/PlayerBugViewer/player_bug_viewer.py.
  • Multilingual viewer syntax check: python -m py_compile Tools/PlayerMultilingualReportViewer/player_multilingual_report_viewer.py.
  • Questionnaire viewer syntax check: python -m py_compile Tools/PlayerQuestionnaireViewer/player_questionnaire_viewer.py.
  • Unity editor upload tester compile check: dotnet build Unity/TH1.Logic.Editor.csproj --no-restore.

Smoke checks when credentials are available:

  • POST a valid current Steam ticket and confirm 200 with all StsCredentials fields.
  • Repeat within 5 minutes and confirm a cache hit path in logs.
  • Test both type: "ossdata" and type: "collectdata".
  • Test action: "steamauth" and confirm later uploads can skip realtime Steam API while cache is valid.
  • Test type: "bugreport" and confirm a unique .zip key under bugreport/.
  • Test type: "multilingualreport" and confirm a unique .zip key under multilingualreport/.
  • Test type: "questionnaire" and confirm a unique .json key under questionnaire/.
  • Attempt wrong objectKey upload with existing credential and confirm OSS rejects it.
  • Attempt oversized upload and confirm client/backend policy rejects it: 3 MB standard, 10 MB bugreport, 512 KB questionnaire.

When cloud deployment is involved:

  • Confirm Function Compute env vars match the intended bucket, role ARN, Steam app ID, Tablestore endpoint, and instance.
  • Confirm the RAM role trust policy allows Function Compute/STS assume-role usage.
  • Confirm the role policy permits the generated bucket path and no broader write path than intended.
  • Confirm Tablestore table Players exists with primary key PlayerId.