How To Create An Aggregator Site

You've probably run into websites that gather content on a single page from a variety of blogs and sites around the internet. Content aggregators, as we call them, keep on earning recognition among publishers and consumers, since they provide a win-win solution for all parties.

To get content for the aggregator, the website owner doesn't have to pay publishers. Usually the only costs of running an aggregator are for technology. Content aggregators give publishers the opportunity to reach a larger audience. And users can find anything they want from a variety of providers on a single website.

So how can you break into the content aggregation market? In this article, we'll talk about great examples of content aggregators and provide some insights on how to craft your own content aggregator platform. But first let's answer the question: What is an aggregator?

What is a content aggregator website?

A content aggregator is a website that collects different content including news articles, social media posts, images, and videos on particular issues from around the web and makes them accessible in one place. Take Techmeme as an example. It's a one-page news aggregator that presents a filtered summary of technology-related articles, reports, and analysis. Another great example is Feedly, a content aggregator that collects feeds from a range of sources all over the internet on a variety of topics including design, marketing, business, politics, and science.

NOTE: Content aggregation is not plagiarism. Plagiarism means using content without permission from the author. Content aggregation is presenting somebody's work with proper credit and a link to the original source.

Let's start a conversation today!

As a client, you might have a lot of questions about our processes, policies, and procedures. Don't hesitate to contact our sales team to communicate and clear things up.

Get in touch →

Content aggregation vs. curation vs. syndication

There's often a serious misunderstanding regarding content aggregation, curation, and syndication, as they're fairly close in meaning. To prevent confusion, let's look at each of these practices in more detail.

Content aggregation is the automatic collection and grouping of content from multiple sources on a particular topic. AllTop provides users with a variety of headlines from popular websites on different topics such as news, tech, sports, entertainment, health, lifestyle, and business.

Content curation is more than just collecting content. Content curation is about selecting the best content that's valuable to the target market and usually including some commentary, context, or views. Great curation is a key element for success. According to Shama Hyder, the CEO of Zen Media, quality content curation means reaching the right audience with the right message at the right time.

Content curation assembles, categorizes, comments on, and presents the most relevant, highest quality information to meet your audience's needs on a specific subject.

- Joe Pulizzi, founder of the Content Marketing Institute

An interesting example of great content curation is the NextDraft app. Dave Pell, creator and editor of NextDraft, summarizes the ten most interesting and important news stories of the day from his own perspective and provides links to the original sources so users can read the full stories.

Content syndication is about republishing content on third-party websites 一 aggregators 一 with links back to the original content. Syndication is the opposite of guest posting, which is publishing unique content on a third-party platform with a brief mention of the author and their blog. Some of the articles on Fox News are syndicated content. When republishing content, Fox News indicates the author and the source where it first appeared.

Content aggregators must provide benefits for news publishers

There are mixed views on the interactions between aggregators and publishers. At first blush, it looks like aggregators provide content producers with new ways to reach target users and subscribers, but in fact not all content aggregators are interested in meeting publishers' needs. Some aggregators don't care that much to promote a publisher's brand, but rather their own. These aggregators don't provide publishers with data on who reads their content, what content users read, and when they consume it, so publishers aren't able to create strong relationships with readers. Content creators may thus have little authority over their content and not own the audience. Additionally, some aggregators choose what content to distribute.

There's still some room for compromise between aggregators and news publishers, however. The secret lies in mutually beneficial cooperation without aggregators trying to take all the credit.

Radhika Shukla, a regional business development director for News Republic, says that to be of benefit for news publishers, content aggregators should provide reach, revenue, recognition, and insights.

Content aggregators should help news publishers reach a wider audience.
Aggregators should produce revenue for publishers.
By partnering with aggregators, publishers should be able to gain recognition for their work.
Content aggregators should provide publishers insights as to what kind of readers consume their content, what articles are most popular, and when users are consuming their content.

Seven great examples of content aggregator websites

Since content aggregators provide convenience, more and more consumers are interested in using these platforms. This is why there are lots of aggregators built each year. Some have already achieved great success and won the hearts of users.

So let's take a look at the best content aggregator websites!

AllTop

Originally set up by Guy Kawasaki, Kathryn Henkens, and Will Mayall in 2008, AllTop is a content aggregator platform that provides users with continuously updated news on any issue. Readers choose topics they're interested in and then see the most recent news headlines from multiple providers on a single web page. Users can position the cursor on a headline and read the first few sentences of the article to get a general idea of what it's about. Additionally, users can create customized pages 一 their own AllTop pages. Users can choose topics and news sources they're interested in to be displayed on their own pages. They can edit their pages any time by adding, removing, or reorganizing feeds.

News feeds are displayed on the AllTop website if the founders of the platform consider them credible, informative, and interesting. And since AllTop uses RSS feed aggregation, the platform captures data only from sources that enable RSS feeds.

Popurls

Created by Thomas Marban in 2005 as a single-page news aggregator, Popurls provides users with a clear picture of what's going on in the world each day. Four years later, Popurls became one of the best online content aggregators and was number five according to the top 50 websites listed in Time magazine.

The Popurls platform has won the hearts of users by providing convenient access to content, smart search, and a no-frills interface. Popurls doesn't just aggregate news feeds from such websites as Reddit, Flickr, del.icio.us, Digg, and many more, but curates them. By simply positioning the mouse over a headline, users can also see the first sentences of an article without being forwarded to a new page.

Feedly

Feedly is one of the best aggregators, offering easy access to news websites, blogs, YouTube channels, and podcasts within one platform. All content is grouped by topics, sectors, and trends. Feedly is now available on iOS and Android and as a web-based service. On the Feedly platform, users can search content by keywords, the name of a website, or an RSS feed address. Users are also able to share any content they like on Facebook, Twitter, Google+, LinkedIn, Pinterest, and Evernote with a single click. Other handy features include tagging, auto-marking as read, read later, and you might also like.

NextDraft

NextDraft is a very interesting case. Created by David Pell, the NextDraft app provides smartly curated news stories wrapped with David's view on things. Each day, David picks the day's ten most captivating news stories and tells readers what's happening in the world by writing what these stories are about. Users can also find links to the original sources to read more.

theSkimm

Created in 2012, theSkimm is an aggregator platform that presents the most important news to start your day. The major events calendar in the application synchronizes with a user's calendar so users know what's happening in politics, sports, the entertainment industry, and other spheres. The application also provides video and audio interviews with people of influence. Once a week, theSkimm presents a ten-minute audio episode that highlights key and relevant topics to users.

Khabar

At Steelkiwi, we have experience in content aggregator development. Khabar is one of the best examples of a news aggregator we've developed. This Saudi platform helps users find the latest news from over 100 news providers. Khabar runs on both iOS and Android and supports two languages, Arabic and English. The application has a minimalist design so users don't get distracted from articles and lets readers share articles they like on Facebook, Twitter, and Google+.

Answers.how

One of the other great content aggregator examples is Answers.how. This aggregator website gives answers to top search queries. All content is grouped by category and subcategory. In addition, readers can use the smart search bar and benefit from smart suggestions or ready-made questions to quickly and easily find information. If no answer is provided to a query, Answers.how pulls data from third-party providers.

How to build a content aggregator website

Here are the principal stages in developing a content aggregator:

Define your type of content aggregator website

The most important step in creating a content aggregator website is to know exactly what content you're going to deliver and how often to update it. There are different types of content aggregator websites that fall into different categories: news aggregators (AllTop, Popurls), review aggregators (Metacritic, Rotten Tomatoes), social network aggregators (Hootsuite, Flock), poll aggregators (FiveThirtyEight, RealClearPolitics), and more. Only after having decided what type of aggregator you want to build and what value you want to deliver can you decide what tools to use.

When starting a content aggregator, it's also important to come up with the right name for your website. A domain name is one of the first things users see and can influence a user's first impressions. We suggest that you keep it short, unique, intuitive, and SEO-friendly and make it easy to spell and remember. Choosing a website name is far from an easy task, but luckily there are lots of domain name generators that can help you with this. If you're interested in using one, read The Top 10 Name Generators to find the best name generator for you.

Identify content aggregator tools and technologies

Since aggregators pull data from third-party sources, you'll need technologies that support this. There are two ways you can extract data from sources 一 via custom web scrapers or web scraping software. Custom web scrapers are built by developers in various programming languages. These aggregator tools and frameworks include Beautiful Soup, Cheerio, and Selenium. For Khabar, we used Scrapy, an open-source Python framework for building a scalable web crawler for extracting data from websites. Web scraping content aggregator software lets you scrape data without being tech-savvy. One of the most famous web scraping solutions are Import.io, Scrapinghub, Listly.io, Octoparse, and Diffbot. The main disadvantage of these tools is that they can't handle large, complex sites.

Create a design that engages readers with content

Many news aggregators, for example AllTop and Feedly, use minimalist design to emphasize content and not distract readers from articles. WikiHow.com and eHow.com adhere to the same principles and use light warm colors on their websites. SmartNews, on the contrary, uses bright colors in the user interface. As you can see, there are no strict principles regarding the design of content aggregator websites. However, we'd suggest creating a minimalist design with a light background and readable text since your main point is news stories.

Build strong filtering features

Since an aggregator is a website with a bunch of data, users should be provided with a strong filter to be able to find the information they need in the shortest possible time. The most common way to facilitate search is by keyword filtering that can be applied to headlines, content, or both. A lot of successful aggregators also logically organize content by splitting data according to categories and subcategories. AllTop, for example, groups content using the hierarchy industries ➝ sectors ➝ websites.

Choose a monetization strategy

It can be difficult to decide how to monetize content curation. Let's see how successful aggregators do this. Feedly charges users for pro memberships and team memberships that provide useful tools such as access to an unlimited number of sources, social sharing functionality, hiding sponsored ads, and Slack integration. Another way to make money is via advertising. You can begin with Google AdSense, a Google program that serves advertisements. You can also integrate native ads and video ads like Flipboard does.

Create a content aggregator website using WordPress or from scratch?

One more important step is deciding how to actually create news aggregator website. There are two common ways 一 from scratch or using WordPress.

WordPress. When time and money is an issue, you might consider building your platform with WordPress. With this option, you don't have to create wireframes and think about design and testing. The costs of developing a website thus are lower compared to a custom product. However, there are some hidden drawbacks of using a ready-made solution. With Wordpress, you aren't able to create a one-of-a-kind product with plenty of features. Additionally, since WordPress is open-source, it's easier for hackers to find vulnerabilities.

From scratch. At Steelkiwi, we create platforms from scratch, as it gives us the opportunity to build unique products with great features and good UI and UX. You can choose any design you want 一 the only limit is your imagination. With custom development you're not limited in functionality. You can add new features whenever you want. In addition, your website will provide great performance and will be protected against hackers.

A few things to remember when crafting a content aggregator

When it comes to building a content aggregator, the process goes beyond the principle stages we've just discussed.

Aggregate, curate, or syndicate?

The majority of content aggregators, like Flipboard and Hacker News, never publish full posts on their websites but instead redirect you to the original source. AllTop and Popurls go further and offer users the first few sentences of an article when positioning the cursor over the headline, however as soon as users click the headline, these platforms also redirect to the original website. Feedly lets users view full articles right on the Feedly website. This platform does include information about the original producer below the headline and allows users to go to the original page by clicking the headline. Answers.how aggregates data from outside sources if there's no answer provided on the Answers.how website.

Aggregation seems to be easier than curation in terms of filling a platform with content. On the other hand, aggregators that curate content are much preferred by users. Remember theSkimm? As of March 2018, this platform already had six million subscribers and had raised $28 million.

Respect the intellectual property rights of publishers

When deciding what content to deliver, it's important to make sure your content, whether it's curated or syndicated, stays on the right side of the law. There are a few things that you need to pay attention to:

Always ask permission;
Avoid displaying content in full;
Don't forget to give credit to the author by backlinking to the original source.

Follow Google algorithm updates

Since a content aggregator deals with different data and links from a bunch of sources, it's necessary to understand Google algorithms and track their changes in order not to be penalized for content duplication. Particularly important Google algorithms for content aggregators include:

Google Panda for duplicated, plagiarized, and thin content;

Google Penguin for spammy and irrelevant links;
Google Hummingbird for low-quality content and keyword stuffing.

Want to build your own content aggregator?

If you want to enter the news market by offering a unique solution, remember that everything you decide and do from the design to the technology stack should be relevant to your core value and target market.

If you're ready to get started or have any questions, feel free to get in touch with us. Meanwhile, you can check out our portfolio to see content management solutions that we've already created.