Real estate listing data is comparable only after property type, transaction type, currency, price period, area unit, location, and observation time are explicit. Preserve the original public listing and displayed values before attempting a cross-market schema.
All Property Scraper is the AgentX multi-source route. Focused Actors cover sources including Zillow, Realtor.com, Homes.com, MagicBricks, NoBroker, and Property24.
Which property entities should stay separate?
- Listing: the source advertisement at a URL.
- Property: the physical unit, parcel, or development the listing describes.
- Offer: sale or rental terms visible in the listing.
- Party: displayed agent, brokerage, owner, or developer.
- Observation: what was visible at collection time.
One property can have multiple listings and offers. One listing can change price or status. Treating URL, property, and offer as the same ID breaks historical analysis.
What belongs in the common schema?
| Group | Fields |
|---|---|
| Provenance | source, listing URL/ID, collected time, query |
| Transaction | sale/rent, displayed status, price, currency, period |
| Property | type, bedrooms, bathrooms, area, area unit, year |
| Location | displayed address, locality, region, postal code, country, coordinates |
| Media | image URLs, floor plan or video references |
| Party | displayed name, role, organization, contact fields when legitimate and necessary |
Keep market-specific fields in an extension object rather than deleting them to fit the common layer.
How should area and price be normalized?
Retain the displayed area text and source unit. Convert square feet and square meters only with a documented formula, and mark values where the source does not specify whether area is usable, built-up, lot, or total.
For rent, separate amount and period. A monthly rent is not directly comparable with weekly rent until the conversion policy is declared. For sale, distinguish asking price from derived price per unit area.
How should the same property be matched?
Exact source listing IDs identify listings, not necessarily properties. Property matching can use normalized address, coordinates, unit number, development name, structural attributes, and image evidence. Assign confidence and preserve alternative candidates.
Avoid merging rural, new-development, or privacy-masked listings based only on approximate coordinates.
What does a useful time series contain?
For each source listing, store:
- first and last observed time;
- every observed price and status;
- material field changes;
- missing/unavailable observations;
- source URL at each point;
- collection query and version.
A disappearing listing is “not observed,” not automatically sold, rented, or withdrawn.
Which questions can the data support?
Public listings can support inventory trends, asking-price analysis, geographic distribution, visible time-on-market proxies, agent/brokerage presence, amenity frequency, and cross-source coverage. They do not directly provide transaction prices, occupancy, property condition, or a complete market census unless another authoritative source supplies those facts.
How should quality be checked?
Review address parsing, currency, transaction type, area unit, coordinates, bedroom/bathroom ranges, duplicate listing IDs, image counts, and source URL reachability. Compare a sample manually in each country because the same field name can represent different concepts across markets.
Use a focused Actor when one market’s special fields are decisive. Use the all-property route for broad discovery and normalized comparison. Continue with How to Validate Scraped Data Quality before scheduling a recurring collection.
Continue in the directory
Turn the guide into a real sample run.
Open the current AgentX contract, check pricing and fields, then validate a narrow output.