RSS, RIS and beyond: how information reaches your watch

Choosing a source is half the work. The other half is deciding how you will receive what it publishes, regularly and without relying on someone visiting the site every morning. Each channel has its own advantages, limits and maintenance cost. This guide reviews the most common ones.

RSS and Atom: the syndication feed

RSS and Atom are XML-based formats a site uses to publish a list of its recent entries: title, link, date and, depending on the publisher, an excerpt or the full text. A reader program checks that list periodically and detects what is new.

Advantages: it is a channel offered by the publisher itself, it has a predictable structure, it includes the publication date and it requires little maintenance.

Limits: many feeds include only the most recent entries, so with infrequent checks you can miss items; some deliver only an excerpt; and not every site offers one.

When it exists, it is usually the first choice.

Sitemaps: the map of the site

A sitemap is a file, usually in XML, in which a site lists its URLs to make the work of search engines easier. It can include the last-modified date of each page, and its location is usually declared in the robots.txt file.

It is useful for discovering new pages on sites without a syndication feed. On the other hand, it provides only URLs, with no title or content; the modification date is not always reliable, and you have to filter by section or by URL pattern.

Detecting changes in pages

Some pages do not publish new entries: they change. A rate table, a list of authorized entities, a regulator’s page of rules in force. For these, the technique consists of saving a version of the page, checking it again periodically and comparing.

It is the only channel for certain content, but also the most delicate:

• A design change or a new notice in the header can trigger false alerts; it is best to watch only the part of the page you care about.
• The alert says that something changed, not what it means. Someone has to interpret it.
• Before automating requests, review the site’s robots.txt and terms of use, and use a moderate frequency.

Email newsletters

A lot of specialized information circulates only by email: newsletters from associations, law firms, public bodies or analysts. The orderly way to bring it in is to use a dedicated mailbox, subscribe that address to the newsletters and process the incoming messages as one more source.

Keep in mind that each newsletter has its own format, that a single message often bundles several news items, and that links frequently go through tracking addresses that hide the real destination. Avoid using a personal mailbox: the source disappears when that person leaves.

RIS, CSV and BibTeX: records you import

Not everything arrives through a continuous channel. Bibliographic databases and reference managers let you export search results in exchange formats:

• RIS: plain text in which each line begins with a short tag identifying the field (document type, authors, title, year, abstract).
• BibTeX: a reference format associated with the LaTeX ecosystem, common in technical disciplines.
• CSV: a text table with values separated by a delimiter. It is universal, but it does not fix which columns must be present, so each import requires specifying which column corresponds to which field.

They are useful for loading a bibliographic search or a historical record. Their weak points: loading is manual and one-off, character encoding and the CSV delimiter easily cause errors, and it is easy to import duplicates if you do not compare by identifier or by title.

Open APIs

An API is an interface designed for a program to query data directly. In the scientific field, several general-purpose open sources offer one: OpenAlex (a catalog of academic works, authors and institutions), Crossref (metadata for publications with a DOI identifier), PubMed (biomedical literature) and arXiv (preprints).

It is the richest channel: it allows queries by topic, author, institution or date and returns structured data. It is also the one that demands the most technical work. Each API has its own data model, terms of use and query limits, which you should read before you start, and it can change versions.

Comparative summary

Channel	Best for	Main caution
RSS and Atom	Media, blogs and bodies that publish entries	Checking often enough
Sitemap	Sites without a syndication feed	Filtering out irrelevant URLs
Change detection	Pages that are updated without publishing entries	False alerts and respecting the site’s rules
Email newsletter	Information that circulates only by subscription	Separating items and resolving links
RIS, BibTeX and CSV	Bibliographic searches and historical records	Duplicates and encoding
Open API	Scientific and structured data	Terms of use and limits

How to decide

1. If the source has an official API or a syndication feed, use it.
2. If not, check whether it publishes a sitemap or an email newsletter.
3. Keep change detection for pages that have no other outlet.
4. Use file import for one-off loads, not as a permanent channel.
5. In every case, keep the link to the original and the capture date.

The access channel is one of the criteria for deciding whether a source makes it onto your list; the others are in how to choose sources.