Usenet news system

Joe Greco
sol.net Network Services

<jgreco@ns.sol.net>

This is a system originally designed for a national network provider, and actually built and deployed for a large (320K subscriber) regional ISP.

Design requirements:

1. Distributed
it must place reader machines near the end-user. This has multiple issues: the reader machines must not eat a large quantity of bandwidth (ideally one can be placed at a T1-connected site and still be able to access a full newsfeed), the clients need to be directed to a topologically close reader machine, etc. The spool servers are carefully placed in strategic network locations with lots of bandwidth.

2. Fault-tolerant
it must be able to survive the crash of various component bits and pieces, while still providing full service (also makes for easier maintenance).

3. Inexpensive
built out of readily-available PC-grade components, much less expensive than heavy "server class" machines. At the same time, this allows "right-sizing" of machines to tasks, and allows for more machines and therefore more redundancy (two inexpensive redundant PC's are much more reliable than one expensive server class Sun, for example, but cost about the same).

4. Scalable
built to handle 320,000 dialup and DSL users, designed to scale linearly at least to the 3,000,000 user mark, and probably substantially beyond.

5. Very Large Scale
designed to store most of Usenet for months, binaries for as long as you can afford the disk (currently > 1TB of Usenet stored on the production system, will shortly be increased to > 2TB). The system in question should fill up at about a hundred million articles.

6. High Bandwidth
dealing with lots of DSL customers pounding the tar out of it, and future concerns about the growth of binaries.

Overview: ISP and Internet consulting and internetworking services, including strategy, design, engineering, implementation, deployment, and ongoing management of virtually any service: your basic one-stop Internet shop.

Specialties: FreeBSD based solutions, large-scale servers, networking, Usenet news, custom systems programming.

Related Interests: Redundancy and reliability through creative deployment of commodity-quality items. (Examples: OSPF combined with a redundant networking infrastructure to guarantee high availability of network, design of server systems that provide server-level redundancy.) Strategy relating to deployment of a service in a regional or national network environment.


Last modified: December 30, 1999 (mk)