Lessons from a year of Postgres CDC in production

(clickhouse.com)

74 points by saisrirampur49 days ago

2 comments

ahachete42 days ago
> "As part of the acquisition, we made an important decision: not only to keep PeerDB open source"> "Here goes the Open Source reference to our validation logic."> "PeerDB Open Source Repository"I hate to be that guy, but PeerDB seems to be governed by the Elastic License [1] which makes it NOT open source.The difference is not small, but significant for many. For one, it won't get integrated into other OSS projects.In my particular case, we have integrated Debezium Embedded into StackGres, as a high level object (CRD) named SGStream [2]. It allows Postgres logical replication from Postgres and exports to another Postgres and/or CloudEvents. No Kafka required. We'd love to consider other alternatives like PeerDB, but not being OSS is a red line we can't cross (having said that, we're really happy with Debezium in general, but having competition and alternatives it's always great).[1]: <a href="https://github.com/PeerDB-io/peerdb/blob/main/LICENSE.md" rel="nofollow">https://github.com/PeerDB-io/peerdb/blob/main/LICENSE.md</a>[2]: <a href="https://stackgres.io/doc/latest/reference/crd/sgstream/#sgstreamspec" rel="nofollow">https://stackgres.io/doc/latest/reference/crd/sgstream/#sgst...</a>[edit: formatting]
- saisrirampur42 days ago
 (this is Sai, the author of the post and also PeerDB co-founder)The wording in that post was an unintentional miss on my part. Apologies for that. We’ve just fixed it. Thanks for flagging it!To add some context, PeerDB was originally released under ELv2 well before the acquisition. During the acquisition we made a choice to keep the project as-is rather than change its license, so this wasn’t a new decision made at that time — just continuity with how the project already existed.We appreciate the feedback, around integration and downstream OSS adoption. That overall makes sense. We’ll take it into account as we think about licensing going forward.Separately, I really wish you tried PeerDB out. The ease-of-use and performance around larger Postgres datasets (TBs to 10s of TB) would’ve been something you would have probably appreciated. That is something we optimized a lot on over that last few years. May be sometime in the future! :)
 - ahachete42 days ago
 Thank you for acknowledging this and updating the blog post correspondingly.I'd love to test and compare PeerDB with Debezium (Embedded), and even SynchDB. But as said, the licensing is a blocker for us. And given the focus and bandwidth we currently have, we won't have the chance to deeply look at it unless there's a high chance we could integrate it into StackGres.Anyway feel free to DM me if you'd like to talk more.
- karlmush42 days ago
 This is a fair point. ELv2 is source-available and doesn’t meet the OSI definition of open source.
- roenxi42 days ago
 > I hate to be that guyI think it is more ClickHouse Marketing being that guy; they have a vaguely aggressive feel to them and slightly-questionable claims like that seem on-vibe to me. Although it is tolerable. Selling databases is hard, the specialists who actually understand the trade-offs are so specialised they usually aren't the person who makes the call on what to use. At least they're selling an interesting [0] DB, Clickhouse has a fun design. They don't mislead anyone who is interested in the details and their documentation is in the end rather detailed.[0] <a href="https://clickhouse.com/docs/academic_overview" rel="nofollow">https://clickhouse.com/docs/academic_overview</a>
 - saisrirampur42 days ago
 Appreciate the feedback here. This wasn’t marketing — just an unintentional miss on my part. <a href="https://news.ycombinator.com/item?id=46392372">https://news.ycombinator.com/item?id=46392372</a>
MuffinFlavored42 days ago
> We discovered that on reconnect, Postgres would start reading the replication slot’s WAL from the restart_lsn—effectively the beginning—rather than from the last processed position. For workloads with long-running, large, or interleaved transactions, this meant unnecessarily re-reading large portions of WAL and drastically impacting replication lag.I wonder if that's intended by Postgres? Doesn't seem logical at first glance.
- saisrirampur42 days ago
 Great question! I believe this behavior is by design in logical decoding. Based on my reading and previous chats with committers, this is my understanding: logical decoding is not stateless, and on reconnection it loses the current transaction state (open transactions, subtransactions, snapshots, catalog state, etc.) that is required for decoding. As a result, a reconnection triggers reading WAL from restart_lsn in order to reconstruct that state.There may be room for improvement in PostgreSQL, by persisting this state to help these reconnections, but I think this is non-trivial and complex than I think, because of the guarantees PostgreSQL must provide around correctness, consistency and reliability.Also, based on what I’ve read, physical replication does not have this issue because it directly ships WAL files (instead of contructing a txn) and reconstructs state on the standby.I’ll let PostgreSQL committers/contributors chime in too on this for a more precise analysis. :)