I worked on Twerq roughly three years ago now and it has been one of the biggest projects I’ve undertaken to date. At its core Twerq enables users to very easily search and find multiple items all in one go. It was one of the first services to “mashup” content, pulling together videos, images and text to help the user to find all the information they could possibly need on a specified search term. In addition to regular searches it also parses rss feeds on some of the worlds most popular news sources such as the BBC, Reuters and CNN.
It features repeat searching, bulk searching, mashups, rss feeds. For me the biggest challenge in this project was creating it so it was both scalable and as feature packed as it could possibly be. Search engine API’s were used a lot in the construction of the project, Google, Yahoo etc. For those items that did not feature API’s I had to use a technique called page scraping or “data mining” to retrieve the desired information for processing. To make this process even more efficienct I implemented caching to reduce bandwidth and overall load on the server.
Unlike FetchMP3, a more recent project, Twerq was not put through a formal closed beta test phase, instead we thoroughly tested at each stage of development to ensure that all was functioning as should be before a public beta.
This website required a combination of skills including:
- XHTML/CSS (WC3 valid)
- Additional Server Module Installation for PHP and Apache etc
Twerq has so many features it would be impossible to explain them all here so check it out at http://www.twerq.com.