When spreadsheets stop scaling: move marketplace data into PostgreSQL
A practical guide to designing your first Spyliner Hub pipeline, from destination access to mappings, write mode, and monitoring.
A spreadsheet can be an excellent operating tool. It becomes a poor raw-data layer when several stores, long histories, dashboards, and internal applications all depend on the same file. That is usually the moment PostgreSQL starts to make sense.
Spyliner Hub connects a marketplace dataset to an existing PostgreSQL table. The important part is not clicking through the wizard—it is making three good decisions about table shape, field mapping, and how repeated records should be written.
Use the right destination
PostgreSQL is useful when the data has several consumers
Choose Database Sync when BI dashboards, transformation jobs, internal tools, or multiple teams need a stable source. Keep Google Sheets when people mainly need a flexible, visible workspace.
For a first pipeline, start with one store and one dataset in a non-critical table. Prove the shape and update behavior before repeating it across every account.
Security before convenience
Create a destination you would be comfortable running in production
- ✓Use a dedicated database role with only the required schema and table access.
- ✓Make the database reachable through an approved network path; do not expose it broadly for convenience.
- ✓Create compatible destination columns for the marketplace fields you actually need.
- ✓Keep connection strings out of tickets, screenshots, source code, and chat messages.
Inside the Hub workflow
Build and test the source-to-table mapping
- 1
Add PostgreSQL
Open Destinations, add PostgreSQL, enter a clear name and connection details, then test and save it.
- 2
Choose the source
Open Syncs, create a sync, and select the connected store and dataset.
- 3
Select the table
Choose the destination, schema, and existing table after Hub inspects its columns.
- 4
Map and test
Match source fields to compatible columns and run Test write access before continuing.
- 5
Set the schedule
Review the configuration, choose the schedule, and create the sync.
The decision that shapes repeats
Use upsert for changing records; insert-only for an append log
Orders and performance records often change after they first appear. Upsert inserts new rows and updates a matching row using a conflict key. The mapped destination columns need a compatible unique constraint or index.
Insert-only leaves existing rows untouched and appends incoming records, subject to PostgreSQL constraints. Use it when that immutable, append-style behavior is intentional—not simply because it sounds safer.
- ✓Map spylinerSyncKey to text or varchar when it is the default record identity.
- ✓Keep the identity column unique when using it as the upsert conflict target.
- ✓Document the write mode and owner so future changes are deliberate.
Quick answers
Questions people usually ask
Does Spyliner Hub support MySQL or a warehouse?+
PostgreSQL is the currently supported database destination.
Is a manual Hub run configured separately?+
No. Manual and scheduled runs use the same saved source, mapping, destination, and write mode.
Which write mode should I use for orders?+
Usually upsert, because status and other order fields can change. Confirm that your conflict key has a compatible unique constraint.
