2 comments

  • sacs0ni27 days ago
    I built this to have a dedicated wire-protocol client for postgres logical replication. General-purpose SQL clients either don&#x27;t implement the replication protocol at all, or bury it behind abstractions designed for other use cases. Replication has a bit different mechanics - it&#x27;s a stateful binary stream requiring LSN tracking, standby heartbeats, and feedback to prevent WAL bloat. Bolting that onto a query-focused client has its own challenges.<p>This is just the transport - raw XLogData frames and LSNs. Use pg_replicate, as an example, if you need &quot;replicate to BigQuery.&quot; Use this if you&#x27;re building replication infrastructure.<p>What it does:<p>- Explicit LSN control - start&#x2F;stop at specific WAL positions for deterministic recovery<p>- Automatic standby feedback - no more forgotten heartbeats filling your disk with WAL<p>- Bounded channels - backpressure propagates to Postgres naturally<p>- Pure Rust, no libpq<p>What it doesn&#x27;t do: pgoutput decoding (intentionally). That belongs in a higher layer. Simplest way of using this:<p>while let Some(event) = client.recv().await? { match event { ReplicationEvent::XLogData { wal_end, data, .. } =&gt; { process(&amp;data); client.update_applied_lsn(wal_end); } _ =&gt; {} } }
    • nkmnz22 days ago
      &gt; Use pg_replicate, as an example, if you need &quot;replicate to BigQuery.&quot; Use this if you&#x27;re building replication infrastructure.<p>Would I use this if I host my own postgres and want to use replication for „real time backups“ into a hot standby?
      • radimm22 days ago
        Not OP - but definitely no. In such a case use just physical or - if you have special use case - logical replication that’s built in.
        • sacs0ni22 days ago
          I agree. pgwire-replication is useful when you need to build a customized and closely controlled pipeline. In fact, it will give you the first part of handling the data (reading from the source), you still need to implement the rest yourself.
    • malodyets24 days ago
      I learned about this tonight when Claude Code picked up your library for my application that uses logical replication. Looking forward to putting it through its paces.
      • sacs0ni24 days ago
        nice! would appreciate any feedback.
    • jsjfusua21 days ago
      [flagged]
  • gunnarmorling21 days ago
    Nice one, great to see this addition to the Rust ecosystem!<p>Reading through the README, this piqued my curiosity:<p>&gt; Small or fast transactions may share the same WAL position.<p>I don&#x27;t think that&#x27;s true; each data change and each commit (whether explicit or not) has its own dedicated LSN.<p>&gt; LSNs should be treated as monotonic but not dense.<p>That&#x27;s not correct; commit LSNs are monotonically increasing, and within a transaction, event LSNs are monotonically increasing. I.e. the tuple commit-LSN&#x2F;event-LSN is monotonically increasing, but not LSNs per se. You can run multiple concurrent transactions to observe this.
    • sacs0ni21 days ago
      Good catch, you are correct. I did mix a few things there and the statements were incorrect or at least very misleading.<p>To demo your point I created a gist, for myself and others to see the (commit-LSN, event-LSN) ordering in action:<p><a href="https:&#x2F;&#x2F;gist.github.com&#x2F;vnvo&#x2F;a8cf59fc3cd8719dbea56d3bb5201f9b" rel="nofollow">https:&#x2F;&#x2F;gist.github.com&#x2F;vnvo&#x2F;a8cf59fc3cd8719dbea56d3bb5201f9...</a><p>I&#x27;ll update the readme to reflect this more accurately. Appreciate you taking the time to point it out.