KLXM Studio – Home
Get in touch

Running a website · Advanced · 1:29 min

Connecting external sources

Fetch RSSFormat for news feeds: a website provides new posts as a list that other programs or websites can fetch automatically.en.wikipedia.org (opens in a new tab), JSONSimple text format for structured data that people can read and programs can process easily. The standard for data exchange via web interfaces.en.wikipedia.org (opens in a new tab), XMLText format for structured data with markup in angle brackets. Many feeds and exchange formats (e.g. RSS, OpenImmo) are based on it.en.wikipedia.org (opens in a new tab) or OpenImmoIndustry standard in German-speaking countries for exchanging property listings between estate agent software and portals or websites.openimmo.de (opens in a new tab) regularly and show it as entries of a data tableStructured content such as team, events, products or references: entries with the same fixed fields that can appear on several pages – often with a detail page per entry..

No sound, with subtitles German and English captionsText of what is spoken in a video, shown in sync. On the web usually a WebVTT file; captions help deaf people and anyone watching without sound.developer.mozilla.org (opens in a new tab) – switch them on and off with “CC” in the player. The video is hosted on this server and only loads when you press play. The steps below are the full text of the video.

Goal

You bring content from an external source into a data table – with field mapping, preview and schedule.

Prerequisites

  • Permission “manage external sources” and the external sources feature (switched on by the network/agency).
  • The address of the source (URL) or a file, e.g. an OpenImmo ZIP from estate-agent software.

Step by step

Transcript: every step matches one subtitle in the video.

  1. Data › Externe Quellen (external sources): RSS feeds, JSON APIApplication programming interface: programs use it to exchange data without the user interface – for example to take over entries from another system.en.wikipedia.org (opens in a new tab), XML and OpenImmo as entries of a data table.
  2. Visitors never load anything from the source: list, detail page, search and API come from your own table.
  3. Templates to start with: RSS/Atom, OpenImmo, JSON API or XML with XPath.
  4. The example “Immobilien” (real estate) reads an OpenImmo file from estate-agent software.
  5. Source: name, format and address – or upload the file on the right. Credentials are stored encrypted.
  6. Fetching: schedule, cache and what happens to entries missing from the source.
  7. Mapping: every table field gets a path in the source – with a conversion, e.g. number or date.
  8. “Probeabruf” (test fetch) shows the first entries as they would arrive in the table – without saving.
  9. On the right: status, OpenImmo file upload and the log of recent fetches. In the list of sources: “Jetzt abrufen” (fetch now).
  10. Imported entries are in “Immobilien” – read-only, because the source maintains them.

Commands & configuration

*/15 * * * * php bin/console sources:sync --all     # Zeitplan zuverlässig per Cronjob

Tips & pitfalls

Tip

The template can create the matching data table with its detail page right away (Tabelle … anlegen).

Tip

Images from the source end up in the media library – re-encoded and without EXIF (collection “Quelle: …”).

Watch out

Do not edit entries from a source by hand – the next fetch overwrites them. Put your own additions in separate fields or tables.

Watch out

Clarify usage rights: content from third-party feeds must not be published automatically.