Asynchrony, Large Data Sets, and Crawling.

Okay, so I come from the bioinformatics world, and I never really cared that much for the concept of of asynchrony.

But I've spent the last week playing with Ning, as part of an architecture designed by Ivan & Pere at Datasalt.com, for peerindex.net; Ning is cool - it turns the nebulous concept of asynchrony into a NATURAL & EXTENSIBLE data acquisition strategy for novice programmers, that know asynchrony is valuable, but don't know why.

Anyways, I'm not going to explain Ning --- because its Java specific , and its very amenable to a google search. Rather, I'm going to explain the 3 important aspects of asynchrony.

- Asynchrony gives you scalability ---> you can start to decouple the ACQUISITION of data from the PROCESSING .

- Asynchrony gives you the FREEDOM to FAIL ----> One or two risky requests, or queries (run at the wrong time of day, for example) won't BREAK everything .

- Asynchrony gives you PERFORMANCE ! ----> By allowing you to process and integrate data in a seperate thread from acquisition , you're able to build applications