5/98 Boardwatch article. Brokered Private Peering (BPP)tm Group (Part I)
Savvis Communications, Williams Communications, Electric Lightwave and Exodus group
An organization dedicated to building a lasting Internet data inter-exchange
architecture built upon a sound commercial foundation that recognizes the value
and importance of high quality, scalable peer-to-peer inter- exchange
bandwidth between Internet Service Providers. Internet Backbone Peering
Author: Michael Gaddis
Contributors: Savvis; Kevin Moss, Electric Lightwave;
Rob Bowman, Exodus; Matthew Bross, Williams;
Tom Nolle, CIMI Corporation
Internet Backbone Peering
Introduction
The commercial Internet was founded upon the same peer-to-peer
interconnection paradigm as the NSF sponsored Internet. During the transition
from the government sponsored Internet to the commercial Internet, National
Internet Service Providers (NSPs) built large national and international
networks that connected to public exchange points located in San Francisco,
Chicago, New York, Washington, D.C., and elsewhere. The public exchange
points (Network Access Point (NAPs) or Metropolitan Area Exchanges
(MAEs)) acted as major data inter-exchange points from 1993 through 1997.
This system is sometimes referred to as the "public peering system."
The public peering system has struggled to keep up with the growth of the
Internet. When the system was established, few rules were created that
governed interconnection. Requirements to provision additional bandwidth to
meet usage growth were not established. Guidelines on which networks should
peer and under what conditions were also not established. The result was a
somewhat ad hoc interconnection system that has, to date, been unable to
adapt effectively to the growth pressures applied to it. In particular, the public
system has not been responsive to the need to create high-quality
business-specific Internet services and Virtual Private Networks (VPNs) built
on the NSP infrastructure.
The public peering system is being gradually augmented with a system of private
interconnections between national providers. A private interconnection is
typically a dedicated DS-3 or OC-3 local loop directly connecting two
networks. This is in contrast to a public peering connection. In public peering, a
network connects to a shared inter-exchange point at the MAE or NAP such
that many peers will be mapped to its network on each link. Private
interconnections are referred to as private peering and the emerging system as
the "private peering system."
Private peering began as a series of local cross connects between large NSPs
at the public MAEs and NAPs to bypass the congested LAN switches that
anchored the public exchange points. The benefits of private peering became
immediately apparent. Private peering provided better control of the peering
process, as well as improved configuration and bandwidth management
between the connected peers.
An obvious outgrowth of the "local" private peering bypass interconnection at
the public MAEs and NAPs was to create peering connections at any mutually
convenient point, to free these peers from any geographical restrictions. As the
bandwidth exchange requirements between large networks grew, new capacity
was installed in geographically distributed cities to allow more efficient Internet
routing. This was the birth of the private peering system as it is found today.
For all its improvement over public peering, the private peering system isn't
perfect. One problem that hinders the transition from public to private peering is
the cost of the additional local loops required to interconnect each network.
For very large networks this cost is easily justified because the connections are
filled with peering exchange data from their true peers. For smaller networks
this cost becomes a barrier to entry to the privately peered network.
Other factors may also limit NSPs' ability to enter into peering relationships.
Private peering has evolved as a rather exclusive club, generally available only
between two NSPs that were clearly peers from the point of view of size
and/or bandwidth exchanged. Smaller NSPs tended to get locked out of the
private interconnection system because of the aforementioned lack of true peer
definitions at the MAEs and NAPs. Over time, this exclusivity has created a
two tiered system at the public MAEs and NAPs with the "good" bandwidth
kept for true peers in private peering and the "not so good" bandwidth (due to
its heavy congestion) left to the shared interconnect of the legacy MAE or NAP
peers.
For many smaller NSPs the quantity and the quality of exchange bandwidth at
the MAEs and NAPs decreased dramatically during 1997 compared to the
overall growth in Internet bandwidth demand. This polarization of bandwidth
quality impacts many users of the Internet, and threatens the diversity in the
Internet that the multiplicity of service providers has created.
For this reason, Savvis Communications, Williams Communications, Electric
Lightwave and Exodus have partnered to sponsor the creation of a Not for
Profit Corporation called "The Brokered Private Peering Group" (The BPP
Group). The BPP Group is chartered to create a private peering system that
will allow any eligible network to provision for and obtain quality inter-exchange
peering bandwidth.
Brokered Private Peering Overview
Brokered Private Peering (BPP) is an Internet peering exchange architecture
whereby three or more peering partners interconnect to an ATM switch in
distributed locations. The peers at a given peering location connect via a DS-3
(minimum), OC-3 or OC-12 loop running multiple virtual channels, one for
each peer. The model for the BPP is to create, via the shared ATM switch, a
shared interconnect with dedicated bandwidth at fractional physical circuit
speeds such that the virtual circuit between two peers operates like a dedicated
local loop connection. With this architecture, all of the benefits of private
peering are achieved without the increased local loop connectivity cost because
one physical loop to the shared switch serves all peers. Furthermore, rules are
established which maintain this effective circuit-level performance over time,
allowing large scale growth in Internet peering bandwidth while maintaining
overall network quality.
The ATM architecture is also suitable for the creation of virtual circuit
interconnection between peers who wish to offer multiple Internet service
grades, or who want to offer VPN services that span NSP networks. The
current NAP/MAE public peering structure has no capabilities in this area, and
it is doubtful that such capabilities could be added.
An important goal of the interconnect is to create a zero loss, low delay, high
availability peering system. Participants are required to manage their bandwidth
professionally and to routinely provision more bandwidth as needed to handle
growth. This requirement to provision dedicated inter-exchange peering
bandwidth is a fundamental requirement of a member of the BPP Group.
The BPP Group also establishes a peering taxonomy to provide a clear
classification of which networks are true peers and which are not. True peers
are required to offer peering to other true peers unless this peering requirement
is waived by the BPP Group governing body. Peers which are classified as
operating within different peer classifications can obtain true peer status by
following inter-class peering requirements. If these requirements are met by a
peer to gain true peer status in another classification group then that group must
offer peering as if it were a true peer within that group. The purpose of the peer
classification system is to establish a sound economic interest for true peers to
exchange bandwidth and to protect true peers from any requirement to peer
with "non-true" peers except on a purely voluntary basis. However, in order to
create a truly egalitarian system, mechanisms exist within the BPP Group for
different peering classes to gain qualifications as a true peer in another class.
The intent of these mechanisms are to provide a fair and reasonable exchange
of bandwidth with built in economic incentives so that an enduring and
adaptable system can be created.
When two peers exceed the bandwidth assigned to their virtual interconnect the
BPP switch provider and the peer participants are expected to (1) increase the
bandwidth on the virtual connection or (2) if the bandwidth exceeds 50% of the
capacity of the local loop transfer the interconnection to a dedicated local loop
or increase the capacity of the local loop into the BPP (i.e. from DS-3 to
OC-3). If a dedicated local loop is used to accommodate the increased
interchange bandwidth between two peers, the peers leaving the physical BPP
are still considered members of the BPP, retaining all applicable rights and
privileges. The bandwidth management model and agreements survive this
private peering transition from shared BPP to dedicated BPP interconnect.
If peers elect to provide special services between their networks (premium
services, VPNs, etc.), the bandwidth for these special services must not impact
the core peering application. Some special services may require dedicated
bandwidth between peers who elect to offer such service; the provisioning of
this bandwidth through the shared BPP switch will be treated as a separate
peer relationship in this case.
The requirement to bilaterally provision peering interchange bandwidth as usage
increases recognizes the importance of the exchange point. Specifically, each
peer has typically spent considerable resources delivering data from their
customers to the exchange point. Discarding data at the exchange point is,
therefore, very wasteful. It is the BPP members belief that peering exchange
points should never be resource overbooked or congested. (This is in marked
contrast to the present public peering system.) Brokered peering becomes,
therefore, a living trust, allowing peer networks to scale their bandwidth
commensurate to their growth into the foreseeable future. This trust eliminates
peering as a barrier to Internet network growth or performance and provides a
level playing field for all Internet carriers to compete.
BPP Group Organization
The BPP Group has not yet been formed as a corporation with legal standing.
This section represents a framework for the future establishment of the BPP
Group corporation.
The organizational structure of the BPP Group consists of a Board of Directors
(BoD), a Technology Advisory Board (TAB), a Director and supporting staff,
BPP exchange point providers, BPP backbone providers and BPP peers. The
BoD governs the operation of the BPP Group and ensures its compliance to
corporate by-laws and corporate mission. The TAB governs the technical
operations of the BPP Group and establishes rules of operation and peering
within the charter of the BPP Group. The Director and staff carry out the
day-to-day operations of the BPP Group under the direction of the BoD and
TAB. The BPP exchange point providers (3 planned) operate the BPP
exchange points under contract to the BPP Group within the specific
operational guidelines established by the BPP Group. The BPP backbone
providers (3 planned) will connect the BPP exchange points with high speed
long haul circuits to provide connectivity between BPP exchange points. The
backbone connectivity will be used to support the ability of a peer to have a
virtual presence at many BPPs with physical connectivity to a few as only one
BPP. Finally, the BPP peers are the members of the organization that utilize the
services of the BPP exchange points or the BPP backbone.
Specific by-laws governing the relationship of all organizational elements will be
detailed at a later date. Membership Criteria
Because of the requirement to bilaterally provision bandwidth as usage
increases, it is important to match true peers at the BPP. Without this careful
matching of interests between peers, it is unlikely that peers will maintain the
quality focus expected of BPP members. This has been the root cause of many
of the peering problems in the public peering system because these real or
perceived differences create different economic incentives to peer (or not to
peer). Therefore, the BPP membership recognizes the differences between
national ISPs (NSPs), regional ISPs and local ISPs.
Additionally, the BPP membership recognizes differences between web-centric
service providers, access- centric and dial-in centric service providers. The
rules of the BPP are designed to match true peers to true peers in a fair manner
to create mutual peering interests that are sustainable over time.
The BPP peering model has a strictly ordered hierarchy based on models of
national, regional and local peering. Networks that qualify for national peering
may peer on any level, regionally qualified peering service providers may peer
at the regional or local level within the territory to which they are assigned.
Networks that qualify for local peering may peer at the local level only.
(However, voluntary peering between any network provider is never
disallowed.)
The initial BPP Group offering will concentrate on NSPs and U.S. national
private peering because that is where the most acute problem lies in the Internet
at the present time. The BPP Group will address other peering requirements of
the hierarchy at a later time. Therefore, membership in the BPP will be limited
to those commercial networks that qualify for national peering classification at
this time. (Expansion of the BPP Group into International connectivity will be
studied by the BPP BoD and TAM at a later time.)
The following two sections will define a primary classification of a service
provider as a national, regional, or local provider with a secondary,
sub-classification type as an access, dial-in, and web-centric provider. Once
those service provider classes are delineated, we define the size criteria
required to be designated a true peer within each sub-classification group.
Finally, the recommended requirements for each class to cross-qualify as a true
peer in another class are detailed so that a network provider can achieve a fully
peered (true peer) status in all classes.
The goal of this classification system is to determine, as objectively as possible,
guidelines for which networks should be compelled to peer and which networks
should not. For those networks not yet big enough to qualify for full peering,
this qualification system provides a roadmap to build to in order to achieve full
multilateral peering qualifications in the future.
"True Peer" Taxonomy and Classification System
Primary Classification
The primary classification system differentiates U.S. national, regional and local
providers. A national provider must possess the following attributes to qualify
for designation as a BPP national peer:
Infrastructure: National network infrastructure (owned or leased) with
ability to provide commercial Internet access or transport to the
commercial customers coast-to-coast in the continental United States.
The typical BPP NSP will have a presence in at least 5 large
metropolitan areas spread coast-to-coast but must have a physical
presence and sell commercial Internet services in at least four of the six
BPP defined regions.
NOC: Professionally staffed Network Control/Operations Center
manned 24 hours a day, 365 days a year.
Competence: Must possess the engineering skills to operate a distributed
Internet peer interchange with multiple providers.
Regional ISPs are providers that do not qualify for national NSP designation
and that span at least two large metropolitan areas in one or more BPP regions.
Local ISPs are those that cannot meet the qualifications as a regional or national
provider. Exceptions to these designation rules can be approved by the TAB
with BoD oversight.
These classification rules only apply to mandatory peering classification.
Voluntary peering is allowed between any two members at any time.
Any commercial network that qualifies for national peering status may join the
BPP Group as a member. Commercial networks designated as regional and
local may not join the BPP Group until the Group opens regional and/or local
peering capabilities. Secondary Classification
All BPP members are sub-classified as one or more of the following service
provider types: Business access oriented, consumer dial-in access oriented or
web-centric. The terms access, dial-in and web-centric will be used to describe
each category respectively.
The secondary classification is used to recommend rules for interconnection that
can equalize the cost/benefit playing field between web-centric and
access/dial-in service providers to provide a sustainable value proposition for
long term bandwidth inter-exchange.
A service provider is considered web-centric if it sends more data then it
receives by a factor of 3-1 through the BPP exchange point. A service provider
is considered an access provider if it is not web-centric and sells at least 25%
(of gross revenue) of its services to businesses or other ISPs at the fractional
T-1 or above speeds. A service provider is a dial-in provider if it sells primarily
(greater than 50% of its gross revenue) as dial-in and/or cable modem Internet
service to consumer oriented customers. A service provider may be designated
as both access and dial-in centric.
The next section will use the primary and secondary classifications to establish
recommended rules for matching true peers to true peers by taking into account
service provider commercial customer size (defined within each category). This
method of defining true peers will then be used in the follow-on section as a
method to define recommended bandwidth inter-exchange rules between true
peers of each service provider class.
Class Based True Peer Classification
Peering is not a welfare system. The size of an ISP matters when comparing
peers to determine "true" peers. When a service provider reaches a certain
critical mass it makes sound economic sense for other true peers to exchange
bandwidth with that peer. However, when a large commercial customer mass
disparity exists between two similar peer types (same primary and secondary
classification) tensions can arise due to a perception that the smaller network is
not providing value to the larger network sufficient to cover the costs associated
with peering. (Free peering is a misnomer, there are many obvious and not so
obvious costs associated with a peering relationship even if there is no money
exchanged for the bartered bandwidth.) The following table defines the
minimum qualifications for each NSP category to be designated as a true peer
in order to qualify as a service provider for multilateral (mandatory) peering
within that classification.
Primary/Secondary
Requirement
True Peer
Status
National access
peer
Minimum of 1,000 revenue
generating business access
circuits at fractional T-1
(minimum of 256 Kbps) or
above speeds. Must have a
presence in at least one BPP
exchange point in all six of the
BPP regions when this size is
reached to remain a BPP
member (the "no lurkers"
clause).
True peer in
national access
class
National dial-in peer
Minimum of 300,000 revenue
generating consumer dial-in,
cable modem or equivalent type
of consumer oriented accounts.
True peer in
national dial-in
class
National web-centric
peer
Minimum of 200 dedicated
computer web-centric host
accounts that collectively "push"
greater than 500 Mbps for more
than 5 hours a day.
True peer in
national
web-centric
class
If a BPP peer does not (yet) qualify for true peer status then the true peers in
their respective categories are not compelled to peer with them. The smaller
BPP peers have the option of (1) negotiating free peering or (2) paying for
peering or (3) buying transiting connections from another network. When a
BPP peer reaches true peer status those connections must be converted to a
true peer relationship using the multilateral peering agreement status afforded to
true peers. The connection through the BPP peering exchange points may be
used for smaller networks to purchase bandwidth from true peers during this
growth and transition period.
Inter-Class True Peer Classification
The following table defines the minimum qualifications for each NSP category
to be designated as a true peer in another class in order to qualify as a service
provider for multilateral (mandatory) peering within that classification. This
mechanism establishes rules for BPP peers in one classification (for example
web-centric peers) to qualify for true peer status in another classification
(access or dial-in peers) so that those peers must honor multilateral
(mandatory) peering with those peers that meet the minimum qualifications.
Primary/Secondary
Requirement
True Peer
Status
National access true
peer
Ability to peer in all six BPP
peering regions.
True peer in
national dial-in
class
Ability to peer in all six BPP
peering regions.
True peer in
national
web-centric
class
National dial-in true
peer
Ability to peer in all six BPP
regions to at least one of the
access true peers BPP
exchange point locations in
each region.
True peer in
national access
class
Ability to peer in all six BPP
peering regions.
True peer in
national
web-centric
class
National web-centric
true peer
Ability to peer in all six BPP
regions to at least one of the
access true peers BPP
exchange point locations in
each region and willingness to
accept regional address
announcements from access
true peers.
True peer in
national access
class
Ability to peer in all six BPP
regions to at least one of the
dial-in true peers BPP
exchange point locations in
each region and willingness to
accept regional address
announcements from dial-in
true peers.
True peer in
national dial-in
class
More than one BPP exchange point may exist in each BPP region. The table
above defines an explicit hierarchy for which true peer class must expend the
effort to reach the other true peer in the peering region in order to gain true peer
status. The BPP Backbone Providers have been commissioned to make this
requirement relatively painless to web-centric and dial-in true peers by setting
up mesh interconnects between BPP exchange points. Through this capability, a
web-centric or dial-in centric national service provider may appear "virtually" at
any exchange point. |
