feat(pgpm/export): loop-based export + auto-discover tables#1137
Open
pyramation wants to merge 1 commit into
Open
feat(pgpm/export): loop-based export + auto-discover tables#1137pyramation wants to merge 1 commit into
pyramation wants to merge 1 commit into
Conversation
Phase 1: Replace ~50 individual queryAndParse() calls in both SQL and GraphQL flows with loops over META_TABLE_ORDER. Both flows now use the same array as the single source of truth, eliminating 2 of 4 update points when adding new tables. Phase 2: SQL flow auto-discovers tables via information_schema.tables for the 3 export schemas. New tables are automatically exported without any config changes. EXPORT_BLACKLIST excludes tables that should not be exported (node_type_registry, blueprint*, etc). GraphQL flow remains config-based (needs field names for query fragments) but the loop-based approach means adding a table only requires updating META_TABLE_CONFIG + META_TABLE_ORDER (2 places, down from 4). Closes constructive-io/constructive-planning#814
Contributor
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Addresses constructive-planning#814.
Phase 1 — Loop-based export: Replaces ~50 individual
queryAndParse()calls in bothexport-meta.ts(SQL) andexport-graphql-meta.ts(GraphQL) with loops overMETA_TABLE_ORDER. Both flows now use the same array as the single source of truth, reducing update points from 4 to 2 when adding a new table.Phase 2 — Auto-discover (SQL flow): After exporting all configured tables, the SQL flow queries
information_schema.tablesfor the 3 export schemas (metaschema_public,services_public,metaschema_modules_public) and auto-exports any new tables not already inMETA_TABLE_ORDER. Field types are auto-inferred frominformation_schema.columnsviamapPgTypeToFieldType().EXPORT_BLACKLISTexcludes tables that should not be exported (node_type_registry,blueprint*).GraphQL flow remains config-based (it needs field names to build query fragments and has no direct DB access), but the loop-based approach means adding a table to GraphQL export only requires updating
META_TABLE_CONFIG+META_TABLE_ORDER(2 places, down from 4).New exports from
export-utils.ts:EXPORT_SCHEMAS— the 3 schemas that participate in exportEXPORT_BLACKLIST— tables excluded from auto-discoverymapPgTypeToFieldType()— now public, used by auto-discoveryNet effect for adding a new table:
META_TABLE_CONFIG+META_TABLE_ORDER(2 places)EXPORT_BLACKLIST(1 place)Review & Testing Checklist for Human
Medium risk — behavioral refactor of export logic, but no new data models.
pgpm exportagainst a real database to verify SQL output matches previous behaviorimage/upload/urloverrides — these only apply to configured tables, auto-discovered ones get standard type inference)EXPORT_BLACKLISTtables are not present in export outputNotes
export-graphql-meta.tsfile went from ~210 lines to ~135 lines (net -80 lines of repetitive calls)export-meta.tsfile went from ~217 lines to ~209 lines, but gained auto-discovery logicMETA_TABLE_ORDER(structural check) instead of regex-matching individualqueryAndParse('table_name')callsuuid,text,boolean,jsonb,int, etc.). Special type hints (image,upload,url) still require an entry inMETA_TABLE_CONFIGwith the overrideLink to Devin session: https://app.devin.ai/sessions/f42f42528fb0465684159227efbef017
Requested by: @pyramation