CDR vs web crawlers: why we run both.
The Consumer Data Right and web crawlers are often pitched as rivals. In reality they're complementary ways to access consented bank data, each with real strengths. Here's how they compare, and why we run both.
Both the Consumer Data Right (CDR) and web crawlers do the same fundamental job: with the customer's consent, they retrieve banking data and bring it into a decision. The customer is always in control, and either way the data flows into the same affordability checks, scores and insights. The difference is in how the data is accessed.
What they have in common
- Consent first. Nothing happens without the consumer's permission.
- Same outcome. Both deliver categorised transaction data that powers balance checks, affordability, scoring and verification.
- Real, current data. Both reflect a customer's actual financial position, not a thin or stale file.
How the CDR works
The CDR is Australia's regulated open banking regime. Accredited data recipients receive data directly from data holders through standardised, secure channels. Its strengths are trust and consistency: strong regulatory safeguards, a predictable data format, and a clear consent framework. Its limitation is reach. Coverage is defined by which institutions and products are available through the regime, and that surface is still expanding.
How web crawlers work
Customer-permissioned web crawlers retrieve the data a consumer chooses to share from their online banking. Their strength is coverage: they can reach institutions, account types and scenarios the CDR doesn't yet cover, which matters when you need to serve as many customers as possible. Their challenge is that they depend on banks' online interfaces, which change over time and need to be maintained.
Why we run both
Treating the CDR and crawlers as an either/or forces a trade-off between regulatory backing and coverage. Run both, and you don't have to choose. TaleFin applies CDR and crawler based on client requirement, behind a single API that returns a consistent result. That delivers the broadest coverage, market-leading reliability, and genuine redundancy: if one pathway is unavailable for a given institution, the other keeps decisions moving.
For lenders and app builders, the practical upshot is simple. You integrate once, and you reach more of your customers, more reliably, with consent at the centre.