PM9311/2/3/5 ETT1CHIP ENHANCED TT1SWITCH FABRIC PM9311/2/3/5
|
|
|
Top Searches for this datasheet
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
PM9311/2/3/5 Enhanced Chip Enhanced Switch Fabric Datasheet
Issue August 2001
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
This document proprietary confidential information PMC-Sierra Inc. Access this information does transfer grant right license this intellectual property. PMC-Sierra will grant such rights only under separate written license agreement. product, process technology described this document subject intellectual property rights reserved PMC-Sierra, Inc.and licensed hereunder. Nothing contained herein shall construed conferring implication, estoppel otherwise license right under patent trademark PMC-Sierra, Inc.or third party. Except expressly provided herein, nothing contained herein shall construed conferring license right under PMC-Sierra, Inc.copyright. Each individual document published PMC-Sierra, Inc. contain additional other proprietary notices and/or copyright information relating that individual document. DOCUMENT CONTAIN TECHNICAL INACCURACIES TYPOGRAPHICAL ERRORS. CHANGES REGULARLY MADE INFORMATION CONTAINED DOCUMENTS. CHANGES INCLUDED FUTURE EDITIONS DOCUMENT. PMC-SIERRA, INC.OR SUPPLIERS MAKE IMPROVEMENTS AND/OR CHANGES PRODUCTS(S), PROCESS(ES), TECHNOLOGY, DESCRIPTION(S), AND/OR PROGRAM(S) DESCRIBED DOCUMENT TIME. DOCUMENT PROVIDED WITHOUT WARRANTY KIND, EITHER EXPRESS IMPLIED, INCLUDING LIMITED IMPLIED WARRANTY MERCHANTABILITY, FITNESS PARTICULAR PURPOSE, NON-INFRINGEMENT. ETT1, Enhanced TT1, TT1, trademarks
2000 PMC-Sierra, Inc. 8555 Baxter Place Burnaby Canada Phone: (604) 415-6000 FAX: (604) 415-6200
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Device Part Numbers
This document contains information ETT1Chip Set, available from PMC-Sierra, Inc. following devices comprise ETT1 Chip Set: Device Name Scheduler Crossbar Dataslice Enhanced Port Processor Part Number PM9311-UC PM9312-UC PM9313-HC PM9315-HC
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Revision History
Issue Number Issue Date March 2000 July 2000 Creation document Added HSTL Power Dissipation values, corrected programming constraints EPP, added OOB, JTAG blocks device dataflow diagrams, added heat sink information Characteristics section, corrected register section, corrected Frame Format tables, added correct definition Interrupt register, corrected diagram Appendix added send receive procedure control packets, added ESOBLCP register explanation, corrected conditions Table corrected Scheduler Refresh Procedure, added drawing Crossbar data flow, corrected token explanation, corrected Figures corrected spelling formatting errors throughout document. Removed from Scheduler Status Register, updated BP_FIFO information Scheduler Control Reset Register, corrected bottom view drawings Scheduler Crossbar, corrected signal description section Scheduler Crossbar power pins, corrected tper3, tpl3 Electrical, added memory address information registers, corrected mechanical drawings, updated state diagram Scheduler, added information about Scheduler timing, updated initialization sequence chips, corrected Dataslice Signal Description ibpen0, added 14(13th 14th Dataslice enable) Control register, updated constraints, added Output Queue Overflow EPP's Interrupt register, updated Output Backpressure Unbackpressure Threshold register, updated LCS2 link synchronization appendix, modified Egress Control Packet Data Format table, Modified ETT1 usage LCS2 protocol section Details Change
August 2001
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Functional Description
Overview
1.1.1 1.1.2 1.1.3 1.1.4 ETT1 Switch Core Features Switch Core Model Protocol (Out-Of-Band)
Architecture Features.
1.2.1 1.2.2 1.2.3 1.2.4 1.2.5 1.2.6 1.2.7 1.2.8 1.2.9 ETT1 Switch Core Basic Cell Flow Prioritized Best-effort Service End-to-End Flow Control Service Subport Mode (2.5 Gbit/s Linecards) Control Packets Redundancy System Configuration Options
Prioritized Best-Effort Queue Model
1.3.1 1.3.2 1.3.3 1.3.4 1.3.5 1.3.6 1.3.7 Unicast Traffic (OC-192). Multicast Traffic (OC-192) Unicast Traffic (Subport Mode). Multicast Traffic (subport mode) Combinations OC-192c Quad OC-48c Linecards Summary Protocol
Explicit Backpressure From oEPP iEPP
1.4.1 Flow Control Crossbar Synchronization
Service
1.5.1 1.5.2 1.5.3 1.5.4 1.5.5 1.5.6 Queues Reservation Tables Frame Timing. Synchronization Configuring Service Changing Source Suggested Sync.
ETT1 Usage Protocol
1.6.1 1.6.2 1.6.3 1.6.4 1.6.5 Protocol Prepend Label Type Fields Control Packets Fields Layer
Out-of-Band (OOB) Interface
1.7.1
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Initialization Procedure
1.8.1 1.8.2 1.8.3 1.8.4 Initial Power-on: Port Board being added:. Scheduler Board being added: Crossbar Board being added:.
Fault Tolerance.
1.9.1 1.9.2 1.9.3 1.9.4 1.9.5 1.9.6 Fault Tolerance Model Soft/Transient Errors Flow Control Refresh Procedure Scheduler Refresh Procedure Refresh Schedulers After Modifying Registers Hard Errors
1.10 ETT1 Signals Interconnections
1.10.1 1.10.2 1.10.3 1.10.4 LVCMOS HSTL Links (A). Live Insertion
1.11 System Latencies
1.11.1 1.11.2 1.11.3 1.11.4 Request Grant Minimum Latency Grant Linecard Cell Latency Cell latency Through Empty System Hole Request Hole Grant Latency
1.12 ETT1 Diagnostics
1.12.1 1.12.2 1.12.3 1.12.4 Device Tests Link Tests Linecard ETT1 Link Diagnostics ETT1 Internal Datapath Tests
Dataslice
Dataslice Blocks.
2.1.1 2.1.2 Interface Control/Status Registers Dataslice Cell Flow
8B/10B Interface Dataslice Registers
2.3.1 2.3.2 Dataslice Summary Dataslice Register Descriptions
Dataslice Signal Descriptions
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Pinout Package Information
2.5.1 2.5.2 2.5.3 Pinout Tables Package Dimensions Part Number
Enhanced Port Processor.
Data Flows Blocks
3.1.1 3.1.2 3.1.3 3.1.4 3.1.5 3.1.6 3.1.7 Data Flows Grant Manager Scheduler Request Modulator Input Queue Manager. Output Queue Manager Output Scheduler Interface Control/Status Registers
Input Dataslice Queue Memory Allocation with Output Dataslice Queue Memory Allocation with Enhanced Port Processor Registers
3.4.1 3.4.2 Enhanced Port Processor Summary Enhanced Port Processor Register Descriptions
Enhanced Port processor Signal Descriptions Pinout Package Information
3.6.1 3.6.2 3.6.3 Pinout Tables Package Dimensions Part Number
Crossbar
Crossbar Blocks.
4.1.1 Interface Control/Status Registers
Modes Operation Access Crossbar Registers
4.4.1 4.4.2 Summary Crossbar Register Descriptions
Crossbar Signal Descriptions
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Pinout Package Information
4.6.1 4.6.2 4.6.3 Pinout Tables Package Dimensions Part Number
Scheduler
Block Structure
5.1.1 5.1.2 5.1.3 5.1.4 Port Block Arbiter Block Interface Control/Status Registers Service
Port State Fault Tolerance. Scheduler Registers
5.4.1 5.4.2 5.4.3 Summary Scheduler Register Descriptions Input Queue Memory (IQM)
Scheduler Signal Descriptions Pinout Package Information
5.6.1 5.6.2 5.6.3 Pinout Tables Package Dimensions Part Number
Characteristics.
Signal Associations Absolute Maximum Ratings Recommended Operating Conditions Electrical Characteristics Electrical Characteristics Timing Diagrams
Appendix
General Packet/Frame Formats
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Frame Formats Dataslice From Enhanced Port Processor. Frame Formats Dataslice From Crossbar. Frame Formats Enhanced Port Processor From Scheduler
Appendix
Common Pinout Configuration
JTAG Interface Reserved Manufacturing Test Pins Power Supply Connections. Example Power Sequence
Appendix
C.1.1
Interfacing Details ETT1
Preventing Underflow
Compensating Round Trip Delay Between Linecard ETT1(LCS) 8b/10b Interface Dataslice
C.2.1 C.2.2 Link Channel
Managing Cell Rates (Setting Idle Count Register) Avoiding Delays Idle Cells
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure
Basic Components Switch Built Around ETT1 Chip Set. ETT1 Switch Core Logical Interconnects Port Configuration ETT1 Switch Queueing Flow Control Frame Concept Fully Redundant Core Simple Redundant Scheduler Configuration Cell Sliced Across Dataslices Crossbars Unicast Ingress Queueing Model Port Ingress Egress Queue Multicast Traffic ETT1 Port Operating Subport Mode with Four OC-48c Linecards Input Request Counters Full Counters Single Priority Request Counters Single Output Share Single Queue Sequence Events Input Side Virtual Input Queues Four Priorities Scheduler Requests/Grants Multiplexed Port Level Each iEPP Single Virtual Output Queue Multicast Cells Multicast Cell Processing EITIBM structure OITIBM structure Scheduler Priorities ETT1 Port Ingress Queues Separate Crossbar Used Convey Backpressure Information iEPPs. Queueing Structure. Ingress Port Logic oEPP Must Treat Transferred Cell Multicast Cell. Single Table Synchronization. Signal Flow. Implementation: Consists Both Local Global Buses Global Device Select Defines Board Number Device Each Board Write Cycle Wait. Write Cycle Wait. Read Cycle Wait Read Cycle Waits. Interrupts Active High Synchronous Redundant Connections Simple Non-Redundant Crossbar Configuration Simple Non-Redundant Scheduler Configuration
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure
Simple Redundant Crossbar Configuration Simple Redundant Scheduler Configuration ETT1 Signals Interconnects Links Control/Status Signals. Request Grant latency ETT1 Event Latencies Cell Latency Illustrating Flow Cells from Linecard ETT1 Loopback Path Control Packet Exchange Between Linecard ETT1 CPU. Testing ETT1 Internal Datapaths Dataslice Data Flow Slicing Data Cell into Logical Dataslices Dataslice CBGA Package Dimensions Side Views Dataslice CBGA Package Dimensions Bottom View ETT1 Port Operating Sub-port Mode with Four OC-48c Linecards Functional Diagram Enhanced Port Processor Enhanced Port Processor CBGA Package Dimensions Side Views Enhanced Port Processor (EPP) CBGA Package Dimensions Bottom View Basic Dataflow Through Crossbar Crossbar CCGA Package Dimensions Side Views Crossbar CCGA package Dimensions Bottom View Scheduler Block Diagram. Port State Machine. Scheduler CCGA package Dimensions Side Views Scheduler CCGA package Dimensions Bottom View Setup Hold 3.3V-tolerant 2.5V CMOS 2.5V CMOS (OOB Interface) Rise Fall Times 3.3V-tolerant 2.5V CMOS 2.5V CMOS (OOB Interface). Test Loading Setup Hold 3.3V-tolerant 2.5V CMOS 2.5V CMOS (JTAG Interface). Rise Fall Times 3.3V-tolerant 2.5V CMOS 2.5V CMOS (JTAG Interface) JTAG Test Loading Setup Hold 3.3V-tolerant 2.5V CMOS 2.5V CMOS (Serdes Interface) Rise Fall Times 3.3V-tolerant 2.5V CMOS 2.5V CMOS (Serdes Interface) Serdes Test Loading Setup Hold Mbps HSTL (EPP/DS Interface). Rise Fall Times Mbps HSTL (EPP/DS Interface) Class HSTL Test Loading Setup Hold Mbps (AIB Interfaces) Rise Fall Times Mbps (AIB Interfaces))
viii
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure Figure
Test Loading Setup Hold 2.5V Mbps PECL (CLK Signals) Clock Period Duty Cycle 2.5V Mbps PECL (CLK Signal) Example Noise Isolation Circuit Example Voltage Divider Voltage Ramp-up Time Available Linecard Cells Striped Across Physical Links 10-bit Data Paths Initial Sequence Expected Port Card Cells Skewed Between Dataslices Idle Counter Delay Outbound Cell Idle Counter Delay Outbound Cell One-deep Queue Outbound Requests
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table
Crossbar Configurations Reservation Table Format from Linecard Switch Definitions Fields from Linecard Switch Format from Switch Linecard Definitions Fields from Switch Linecard Encoding 4-bit Type Fields Usage Ingress Request Label_1 Usage Egress Grant Label_1 Usage Egress Payload Label_2 Control Packet Format Control Packet Format Request Count Control Packet Format Start/Stop Control Packet Format Fiber Gbit/s Channel Interface Configurations Mapping Segments Gbit/s Channels Control Signals Multicast Group Selects Unicast Group Selects Refresh-sensitive Registers Scheduler Dataslice Register Summary Dataslice Signal Descriptions Dataslice Pinout (left side) Dataslice Pinout (right side) Dataslice Alpha List Dataslice CBGA Mechanical Specifications Linecard Request Count Maximum Values Input Dataslice Queue Memory Allocation Output Dataslice Queue Memory Allocation Egress Control Packet Data Format Dataslice Addressing Enhanced Port Processor Register Summary Enhanced Port Processor Signal Descriptions Enhanced Port Processor Pinout (left side) Enhanced Port Processor Pinout (center) Enhanced Port Processor Pinout (right side) Enhanced Port Processor Alpha List Enhanced Port Processor CBGA Mechanical Specifications Crossbar Register Summary Crossbar Signal Descriptions Crossbar Pinout (left side) Crossbar Pinout (center)
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table Table
Crossbar Pinout (right side) Crossbar Alpha List Crossbar CCGA Mechanical Specifications Non-TDM Traffic Scheduler Register Summary Scheduler Signal Descriptions Scheduler Pinout (left side) Scheduler Pinout (center) Scheduler Pinout (right side) Scheduler Alpha List Scheduler CCGA Mechanical Specifications Signal Associations Absolute Maximum Ratings Absolute Maximum Ratings 3.3V-tolerant 2.5V CMOS (OOB Serdes Interface) Absolute Maximum Ratings 2.5V CMOS (JTAG Interface) Absolute Maximum Ratings Mbps HSTL (EPP/DS Interface) Absolute Maximum Ratings Mbps (AIB Interface) Absolute Maximum Ratings 2.5V Mbps PECL (CLK Signals) Recommended Operating Conditions Additional Power Design Requirements Electrical Characteristics 3.3V-tolerant 2.5V CMOS (OOB Interface) Electrical Characteristics 2.5V CMOS (JTAG Interface) Electrical Characteristics 3.3V-tolerant 2.5V CMOS (Serdes) Electrical Characteristics Mbps HSTL (EPP/DS Interface) Electrical Characteristics Mbps (AIB Interfaces) Electrical Characteristics 2.5V Mbps PECL (CLK Signals) Electrical Characteristics 3.3V-tolerant 2.5V CMOS (OOB Interface) Electrical Characteristics 2.5V CMOS (JTAG Interface) Electrical Characteristics 3.3V-tolerant 2.5V CMOS (Serdes Interface) Reference Clock Mbps HSTL (EPP/DS Interface) Electrical Characteristics Mbps (AIB Interface) Electrical Characteristics 2.5V 200Mbps PECL (CLK Signals) Jitter Static Phase Offset Request/Data from Frame Format Data from Frame Format Grant/Data Frame Format Control Link Format Dataslice Crossbar Frame Format Crossbar Dataslice Frame Format Scheduler Frame Format Scheduler Frame Format
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Table Table Table
Suggested 8b/10b Decode Within Dataslice Dataslice Egress Truth Table Programmed Token Delay Inter-link Skew
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
xiii
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Functional Description
OVERVIEW
ETT1Chip provides crossbar-based switch core which capable switching cells between ports with each port operating data rates Gbit/s. This section describes main features switch core cells flow through complete system that based ETT1 Chip Set. This document often refers port rates OC-192c OC-48c. ETT1 Chip itself operates fixed cell rate cells second port thus unaware actual data rate attached link. switch might ports OC-192c, could ports Gbit/s Ethernet; internal cell rate that determined ETT1 Chip Set, link technology.
1.1.1
ETT1 Switch Core Features
ETT1 switch core provides following features: Gbit/s aggregate bandwidth ports Gbit/s bandwidth each Each port configured OC-48c OC-192c Both port configurations support four priorities best-effort traffic unicast multicast data traffic support guaranteed bandwidth zero delay variation with Mbit/s channel resolution LCSprotocol supports physical separation switch core linecards feet Virtual output queues eliminate head-of-line blocking unicast cells Internal speedup provide near-output-queued performance Cells transferred using credit mechanism avoid cell losses buffer overrun In-band management control Control Packets Out-of-band management control dedicated interface Optional redundancy shared components fault tolerance Efficient support multicast with cell replication performed within switch core
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
1.1.2
Switch Core Model
ETT1 Chip designed provide switch core, complete packet switch system. complete switch consists more switch cores, together with number linecards. Each linecard connects port core. linecard includes physical interface (fiber, co-axial cable) transmission system such SONET/SDH Ethernet. linecard analyzes incoming cells packets determines appropriate egress port priority. linecard contains cell/packet queues that needed allow transient congestion through switch. ETT1 switch core operates fixed size cells. linecard transmission system uses variable length packets, cells size different from those used core, then linecard responsible performing segmentation reassembly that needed. Figure illustrates this generic configuration.
Figure Basic Components Switch Built Around ETT1 Chip Set.
ETT1 Switch Core port Linecard SONET/SDH Ethernet. ETT1 Chip
Linecard SONET/SDH Ethernet. port
Protocol
ETT1 Chip been designed allow feet meters) physical separation between ETT1 core linecard. LCSTM(Linecard Switch) protocol used between ETT1 core linecard ensure lossless transfer cells between entities. However, while protocol must implemented, physical separation mandatory; linecard could reside same physical board ETT1 port devices. switch core main interfaces. interface between linecard core port, described Protocol section.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
second interface between ETT1 devices local CPU. ETT1 Chip requires local configuration, diagnostics maintenance purposes. single control complete ETT1 core common Out-Of-Band (OOB) bus. ETT1 devices have interface bus. described Section 1.1.4 "The (Out-Of-Band) Bus" page
1.1.3
Protocol
Linecard-to-Switch (LCSTM) protocol provides simple, clearly defined interface between linecard core. this section introduce LCS. There aspects LCS: per-queue, credit-based flow control protocol physical interface
protocol provides per-queue, credit-based flow control from ETT1 core linecard, which ensures that queues overrun. ETT1 core shallow cells) queues both ingress egress directions. These queues compensate latency between linecard core. think these queues simply extensions queues within linecards. queues themselves described further Section "Prioritized Best-Effort Queue Model" page protocol asymmetrical; uses different flow control mechanisms ingress egress flows. ingress flow uses credits manage flow cells between linecards ETT1 core. core provides linecard with certain number credits each ingress queue core. These credits correspond number cell requests that linecard send core. each cell request that forwarded given queue core linecard must decrement number credits that queue. core sends grant (which also credit) linecard whenever core ready accept cell response cell request. some later time, which dependent complete traffic load, cell will forwarded through ETT1 core egress port. egress direction linecard send hole requests, requesting that ETT1 core does forward cell celltime. linecard issue hole request each four best effort unicast multicast priorities. linecard continually issued hole requests four priorities then ETT1 core would forward best effort traffic linecard. protocol information contained within eight byte header that added every cell. physical interface that been implemented ETT1 Chip based faster version Gigabit Ethernet Serdes interface, enabling off-the-shelf parts physical link. This interface provides Gbit/s serial link that uses 8b/10b encoded data. Twelve these links combined provide single link operating Gbaud, providing effective data bandwidth that excess OC-192c link. NOTE: protocol defined "LCS Protocol Specification Protocol Version available from PMC-Sierra, Inc. This version supersedes Version Version first supported Chip with Enhanced Port Processor device (also referred ETT1 Chip Set) will supported future PMC-Sierra products. ETT1 implementation protocol described further Section "ETT1 Usage Protocol" page
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
1.1.4
(Out-Of-Band)
ETT1 Chip requires local perform initialization configuration after Chip reset core configuration changed perhaps ports added removed, example. provides simple mechanism whereby local configure each device. Logically, provides address/data with read/write, valid ready signals. purpose maintenance diagnostics; involved per-cell operations.
1.2.1
ARCHITECTURE FEATURES ETT1 Switch Core
ETT1 switch core consists four types entities: Port, Crossbar, Scheduler, CPU/Clock. There more instances each entity within switch. example, each entity might implemented separate PCB, with each interconnected single midplane PCB. Figure illustrates logical relationship between these entities.
Figure ETT1 Switch Core Logical Interconnects
Crossbar Linecard Port Port Linecard
Flow Control Crossbar
Scheduler
CPU/clock
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Each ETT1 port attached more linecards. port contains shallow cell queues implements protocol. port Scheduler exchange information about cells that waiting forwarded through Crossbar core. Scheduler maintains local information number cells that waiting ingress egress queues. arbitrates amongst cells ingress queues, instructs ports which cell they forward through Crossbar each cell time. Crossbars used. first, referred simply `the Crossbar', interconnects ports with other ports, enabling cells forwarded from ingress port queues egress port queues different port). Crossbar reconfigured every cell time provide non-blocking one-to-one one-to-many mapping from input ports output ports. Each Crossbar port receives configuration information from attached port; Crossbars communicate directly with Scheduler. second Crossbar flow-control Crossbar. passes output queue occupancy information from every egress port every ingress port. ingress ports this information determine when requests should should made Scheduler. CPU/clock provides clocks cell boundary information every ETT1 device. also local which read write state information every ETT1 device bus. CPU/clock entity necessary element ETT1 switch core, does contain ETT1 devices.
1.2.2
Basic Cell Flow
ETT1 Chip consists four devices. Their names (abbreviations) are: Dataslice (DS) Enhanced Port Processor (EPP) Scheduler (Sched) Crossbar (Xbar)
This section describes cells flow through these four devices.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure Port Configuration ETT1 Switch
Input Data Slice
Crossbar
Output Data Slice
Input
Flow Control Crossbar
Output
Scheduler data cell flow control flow
Figure shows two-port configuration ETT1 switch. Only ingress queues left hand port egress queues right hand port shown. port either seven devices. contains cell queue memory also Serdes interface linecard. single cell "sliced" across Dataslice devices, each which manage slices. port controller, determines where cells should stored memories. Multiple Crossbar devices make full Crossbar. single Scheduler device arbitrate entire core. cell traverses switch core following sequence events: cell request arrives ingress port, passed EPP. adds request other outstanding cell requests same queue. some later time, issues grant/credit source linecard, requesting actual cell specific queue. linecard must respond with cell within short period time. cell arrives ingress port header passed EPP. determines destination queue from header, then tells Dataslices where store cell (each Dataslice stores part cell). also informs Scheduler that cell arrived Scheduler should list cells waiting forwarded through Crossbar. modifies label replacing destination port with source port, egress port linecard which port sent cell. Scheduler arbitrates among queued cells sends grant those ports that forward cell. Scheduler also sends routing each destination (egress) ports; this
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
tells ports which many source ports will sending cell. source sends read command Dataslices, which then reads cell from appropriate queue sends Crossbar. same time, destination ports send routing information Crossbar. This routing information used configure internal connections within Crossbar duration cell time. cell then flows through Crossbar from source port destination port. cell arrives destination port, receives header cell. uses this information decide which egress queue cell should stored. this multicast cell which caused egress multicast reach occupancy limit, then would send congestion notification Scheduler. some later time, decides forward cell linecard. sends read command Dataslices which read cell from memory forward cell linecard. egress also sends flow control ingress EPP, informing that there exists free space more egress EPP's output queues. Also, transmitted cell multicast cell then this cause egress queue from full full, which case notifies Scheduler that (the EPP) once again accept multicast cells. above description does account interactions that take place between ETT1 devices, describes most frequent events. general, users need aware detailed interactions, however knowledge main information flows will assist gaining understanding some more complicated sections.
1.2.3
Prioritized Best-effort Service
ETT1 switch core provides types service. first prioritized, best-effort service. second provides guaranteed bandwidth described later. best-effort service very simple. Linecards forward best-effort cells ETT1 core where they will queued. Scheduler arbitrates among various cells; arbitration algorithm dual goals maximizing throughput while providing fair access ports. more than cell destined same egress port then Scheduler will grant cells others will remain their ingress queues awaiting another round arbitration. service best-effort that Scheduler tries best satisfy queued cells, case contention then some cells will delayed. Scheduler supports four levels strict priority best effort traffic. Level cells have highest priority, level cells have lowest priority. level cell destined given port will always granted before cell different priority level, same ingress port, that destined same egress port. `flow' sequence cells from same ingress port same egress port(s) given priority. Best-effort flows either unicast flows (cells flow only egress port), multicast flows which case cells many, even all, egress ports).
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
1.2.4
End-to-End Flow Control
full queueing flow control model shown Figure NOTE: Credits backpressure used every transfer point ensure that cells cannot lost lack buffer space.
Figure Queueing Flow Control
ETT1 core
Linecard
ETT1 port
ETT1 Crossbar
ETT1 port
Linecard
Ingress queues credits
Ingress queues
Egress queues hole requests
Egress queues
ETT1 backpressure
cell flow backpressure/credits
1.2.5
Service
ETT1 service provides guaranteed bandwidth zero cell delay variation. These properties, which available from best-effort service, mean that service might used provide ACBR service, example. ETT1 core provides service same time best effort service, cells integrate smoothly with flow best-effort traffic. effect, cells appear Scheduler cells highest precedence, even greater than level zero best-effort multicast traffic. service operates enabling ETT1 port (and linecard) reserve crossbar fabric some specified cell time future. Scheduler notified this reservation will schedule best-effort cells from ingress port egress port during that cell time. Each port make separate reservations according whether will send and/or receive cell each cell time. Several egress ports receive same cell from given ingress port; therefore, service inherently multicast service. order provide guaranteed bandwidth over long period time, ingress port will want repeat reservations regular basis. support this ETT1 core uses internal construct called
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Frame. Frame simply sequence ingress egress reservations. length (number cell times) Frame configurable maximum 1024 cells. Frame repeats after certain fixed time. ports synchronized start Frame, operate cell-synchronously with respect each other. Thus, cell time, every ETT1 port knows whether made reservation send receive cell. application note "LCS-2 Service ETT1 Switch Core", available from PMC-Sierra, Inc. Figure illustrates idea Frame. Frame slots 1024 less) where each slot 40ns cell time. Frame repeated continuously, adjacent Frames separated guard band least cells.
Figure Frame Concept
40ns
Frame
Guard band
ETT1 ports must synchronized with respect start Frame. This synchronization information distributed Scheduler. Scheduler generate synchronization pulses itself, receive "Suggested Sync" pulses from ETT1 port which turn receive synchronization signals from linecard. linecards need exactly synchronized either ETT1 core other linecards. protocol will compensate asynchrony between linecard ETT1 core.
1.2.6
Subport Mode (2.5 Gbit/s Linecards)
ETT1 switch core have ports. Each port supports linecard bandwidth excess Gbit/s. This bandwidth might used single linecard (for example, OC-192c), ETT1 port configured share this bandwidth among four ports, each Gbit/s. This latter mode, specifically four Gbit/s linecards, referred subport mode, sometimes quad OC-48c mode. single ETT1 switch have some ports `normal' mode, some subport mode; there restriction. four levels prioritized best-effort traffic available configurations. However, there some important differences between modes. difference that header must identify subport associated with each cell. bits label fields used this purpose, described below. second difference that must carefully manage output rate each subport, overflow buffers destination linecard, this also described below.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
third difference that must maintain separate request counters each subports. must also maintain separate egress queues. number queues increase four-fold order preserve independence each subport. Section "Prioritized Best-Effort Queue Model" page describes various queuing models great detail.
1.2.6.1
Identifying Source Subport
manages single physical stream cells cells/second. subport mode must look each incoming cell determine which subport sent cell. label field used achieve this. bits within label (referred Specification bits) used denote source subport, numbered through bits must inserted label before cell arrives EPP. bits might inserted source linecards themselves. Alternatively, they might inserted four-to-one multiplexer device placed between subport linecards EPP. this latter case, multiplexer device might able re-calculate CRC. maximum flexibility, configured calculate either with without bits.
1.2.6.2
Egress Cell Rate
Within Output Scheduler process which different from, should confused with, ETT1 Scheduler device. every OC-192 cell time, Output Scheduler looks egress queues decides which cell should forwarded attached egress linecard(s). subport mode, Output Scheduler will constrain egress cell rate overflow Gbit/s links. does this operating strict round-robin mode, that OC-192 cell time will only send cell subports. four Gbit/s subports labeled through some cell time Output Scheduler will only send cell from egress queues associated with subport next time, will only consider cells destined subport etc. cell time, there cells sent selected subport, then Idle (empty) cell sent. example, subports connected linecards subports disconnected, then Output Scheduler will send cell then empty cell, then cell then empty cell, then repeat sequence. effective cell rate transmitted each subport will exceed 6.25 cells second.
1.2.7
Control Packets
protocol provides in-band control packets. These packets (cells) distinct from normal cell traffic that they pass through fabric egress linecard, intended cause some effect within switch. There classes Control Packets. first class, referred Control Packets, exchanged between linecard ETT1 (via Dataslices). intention that Control Packets form basic mechanism through which linecard ETT1 exchange information. This simple mechanism subject cell loss, should supplemented some form reliable transport protocol that would operate within ETT1 linecards. second class, referred Control Packets, used manage link between linecard ETT1 port. These Control Packets used start stop flow cells link,
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
provide synchronization event information, recover from grant/credit information that lost cells corrupted transmission.
1.2.7.1
Sending Control Packet from Linecard
Before sending (OOB linecard) control packet, must first write control packet header payload data into appropriate locations Dataslice. (See Section "Output Dataslice Queue Memory Allocation with EPP", Table page 164.) header customer-specific control packets should written into Dataslices shown, payload data completely customer. send control packet which been written into Dataslice queue memory, writes ESOBLCP register with control packet type select bits [5:4] (see bit-breakout), linecard fanout bits [3:0]. port connected subport linecards, then bits [3:0] subport-bitmap. port connected OC-192c linecard, then must when wishes send When been sent linecard(s) indicated bits [3:0], bits [3:0] will read back Since control packets have higher priority than other traffic type, they will sent immediately, unless programmed send only idle cells.
1.2.7.2
Sending Control Packet From Linecard
linecard sends control packets using regular request/grant/cell mechanism. (linecard OOB) control packet must have request label (See Section 1.1.3 "The Protocol" page 17.) When receives cell payload control packet, stores cell Dataslices' Input Queue memories raises "Received LC2OOB/CPU Control Packet from Linecard." interrupt. OC-192c mode, will always Linecard input queue control packets from each linecard only cells deep, soon sees "Received LC2OOB." interrupt, should read appropriate "Subport* FIFO Status" register (0x80.0x8c). [3:0] that register will tell many control packet cells currently input queue; bits will tell offset head 8-cell input queue. That queue offset should used form addresses Dataslices' Input Queue memories. Section "Input Dataslice Queue Memory Allocation with EPP" page 162, Dataslice Input Queue memory addressing. Then should read those addresses obtain payload data. When read payload data, should write appropriate "Linecard FIFO Status" register (any value). write that register, regardless write data, will cause head queue dequeued, freeing that space input queue. Section 1.6.3 "Control Packets" page more details.
1.2.8
Redundancy
ETT1 core configured with certain redundant (duplicated) elements. fully redundant core capable sustaining single errors within shared device without losing re-ordering cells. This section describes main aspects redundant core. complete switch have ETT1 cores,
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
with linecards dual-homed both cores, while this valid configuration ETT1 core does provide specific support such configuration. Figure shows fully redundant core. Crossbars Scheduler fully replicated, links connecting those devices. port devices (EPP Dataslices) replicated. important understand that Dataslice fails incurs transient internal error), then data lost, only within that port.
Figure Fully Redundant Core
Crossbar
Dataslice Crossbar
Dataslice
Flow Control Crossbar Port Flow Control Crossbar
Port
Scheduler
Scheduler-
Fault tolerant region
fully redundant core operates exactly same non-redundant core. Consider EPP: sends request information both Schedulers. Schedulers operate synchronously, producing identical outputs, information (grants) received will same. Dataslices operate exactly same with their Crossbar devices.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
information carried links protected checksums. example, receives information with checksum error link, then simply ignores that link uses information from other link. errors reported local interrupts. redundant core tolerant such single errors, provided that they detectable. general, errors that occur links within Scheduler Crossbar will detectable. should clear from Figure that only Dataslice have redundant connections. individual Crossbar Scheduler cannot compensate information that received with checksum error. following discussion reviews these types devices affected receiving corrupted information. Crossbar detects checksum error then simply marks cell being corrupted. This information passed egress Dataslice which will ignore that link information from other Crossbar. Scheduler more complicated because must maintain accurate state information occupancy queues well backpressure status some egress queues. Scheduler receives checksum error then must effectively remove itself from system. Consider simple configuration illustrated Figure Scheduler sees checksum error information receives from Scheduler incorrect state information. immediately disables outbound links that know that Scheduler should ignored. EPPs only accept information from Scheduler
Figure Simple Redundant Scheduler Configuration
Scheduler
Scheduler
Scheduler-0 board inserted system, then must have internal state updated match that Scheduler-1. This done using sequence actions that controlled ETT1 CPU. essence, linecards temporarily stopped from sending cells, state within each downloaded into Schedulers, then linecards restarted. entire process very rapid (much less than 1ms), does cause best-effort traffic suspended brief time.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
1.2.9
System Configuration Options
Most previous sections have assumed particular switch configuration consisting ports OC-192c, with byte payload cells full redundancy. ETT1 Chip four aspects that configured according user's requirements. these aspects have been described: quad OC48c versus single OC-192c port, redundant system versus non-redundant system. other aspects described this section. NOTE: four aspects orthogonal choice particular aspect does limit choices other three aspects.
1.2.9.1
Payload: bytes bytes
ETT1 core protocol designed forward fixed length cells. Each cell consists header bytes) payload. size this payload either bytes bytes. choice payload size function linecard traffic: 53-byte Acells probably byte payload; fragmented packets might obtain greater efficiency with byte payload. This flexibility cell size ETT1 Chip slices cells. Figure shows cell sliced across Dataslices Crossbars.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure Cell Sliced Across Dataslices Crossbars
ETT1 Port Cell Header Bytes) Dataslice Slices Crossbar Crossbar Crossbar Crossbar
ETT1 Port
Dataslice Slices
Dataslice Slices Payload Bytes)
Dataslice Slices
Dataslice Slices 10,11
Crossbar Crossbar Crossbar Crossbar
Dataslice Slices 10,11
Extra bytes bytes slice)
Dataslice Slices 12,13
Dataslice Slices 12,13
byte payload cell requires Dataslices port Crossbars port, non-redundant). byte payload cell requires seven Dataslices port Crossbars. capable supporting seventh Dataslice. Because extra data obtained slicing cell then this does affect clock frequency Chip Set. only effect that link between ETT1 port linecard must from Gbaud Gbaud. This done making link wider, faster.
1.2.9.2
ports
ETT1 core support maximum ports. Smaller configurations supported simply using some ports. However, maximum system eight port configuration, then some
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
cost savings achieved using fewer Crossbar devices. Crossbar configured 32-port core, shown Table
Table Crossbar Configurations
Number OC-192c ports
Number Crossbars required (non redundant) Flow Control Flow Control Flow Control
Number Crossbars required (redundant) Flow Control Flow Control Flow Control
reduction number crossbars achieved having single port more than connection each Crossbar. port system, each port will connection each Crossbar device; port system, each port will four connections each Crossbar. (This DOES apply Flow Control Crossbars; each port will always only connection.)
PRIORITIZED BEST-EFFORT QUEUE MODEL
ETT1 switch core been designed maximize cell throughput while keeping latency small providing "fair" throughput among ports. ETT1 core supports four levels strict priority. Level highest priority level lowest. conditions being equal, core will always forward higher priority cell before lower priority cell. Furthermore, core will fair inputs. This fairness very difficult quantify only meaningful over time periods many cells. Over short time periods (tens perhaps hundreds cells) there temporary unfairness between ports. cell lower priority pass cell higher priority under certain conditions: Since scheduling pipeline shorter lower priorities, cell lower priority emerge from switch sooner than cell higher priority lower-priority cell sent immediately after higher-priority cell. Hole requests higher priority allow cells lower priority pass cells higher priority. linecard responds grants faster than required meet system round trip time constraint, then multicast cells lower priority pass unicast cells higher priority, (programmable) delay EPP's "Internal Delay Matching Adjustments" register. subport mode only: multicast output queues (VIQs) shared subports, when subport oversubscribed extended period time, blocking result subports. This blocking multicast allow unicast cells same priority pass. this section describe queueing model provided ETT1 switch core this prioritized best-effort traffic. start considering only OC-192c ports. following sections describe model differs OC-48c ports.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
1.3.1
Unicast Traffic (OC-192)
Every ETT1 port separate request counter ingress queue each (output port, priority) combination which send unicast cells. full configuration, ETT1 port send ETT1 ports (including itself), four priorities, will have unicast request counters ingress queues. these counters queues independent that switch does suffer from head-of-line blocking1. Figure illustrates unicast ingress queueing model port.
Figure Unicast Ingress Queueing Model Port
ingress queues referred Virtual Output Queues (VOQs). This name indicates that each queue holds cells going just output port. Every ETT1 port also same number unicast egress queues; queue every (input port, priority) combination that send cells port. Every port will have unicast egress queues. acceptable have single egress queue priority, this lead gross unfairness between ports. These egress unicast ports called Virtual Input Queues (VIQs) each queue only holds cells that have come from single input port. useful model think being directly connected through Crossbar single VIQ. cell head waiting scheduled such that move tail egress port. associated ingress informed whenever egress queue full. this case, ingress port will issue requests Scheduler until sufficient cell space becomes available egress queue. transfer cells from ingress egress queues lossless from queueing perspective. best effort unicast ingress egress queues store cells.
1.3.2
Multicast Traffic (OC-192)
ETT1 core also supports multicast cells. Multicast cells cells that must forwarded more egress ports. header will indicate that cell multicast, will also contain multicast group identifier. ETT1 port uses this identifier determine list ports which cell should This list ports called multicast fanout. Different multicast groups will probably have different fanouts. Scheduler arbitrates among unicast multicast cells. multicast cell given priority will always have priority over unicast cell same priority, else being equal. Physical cell replication takes place within Crossbar.
Head -of-line blocking phenomenon encountered input queued switches case where cell destined output port delayed because cell head same FIFO queue destined some other output which currently congested.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Scheduler will necessarily schedule ports fanout same cell time. other words, same multicast cell might pass through Crossbar several occasions before egress ports have received their copy cell. Every ETT1 port single ingress queue single egress queue multicast traffic each priority, shown Figure (There also request counters, unicast cells, these shown.)
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure Ingress Egress Queue Multicast Traffic
Scheduler maintains information occupancy multicast ingress queues, receives backpressure/unbackpressure signals multicast egress queues. egress multicast queue backpressured then there question what happens cell head ingress queue that trying send that backpressured egress queue well other ports). Scheduler blocks ingress queue, then system will experience head-of-line blocking multicast cells that priority. Every ETT1 port select wants handle multicast cells that sent each four multicast egress queues, local ETT1 either enable disable backpressure signal from being sent Scheduler. NOTE: This selection egress queues, ingress queues. port that multicast backpressure disabled, then that port will never cause head-of-line blocking occur ingress multicast queue that will send port that multicast backpressure enabled, then that port assert backpressure signal egress queue gets full. Consequently, this might cause head-of-line blocking multicast ingress queues that send cells this port. Therefore, head-of-line blocking only guaranteed occur ports that receive multicast cells have their multicast backpressure signals disabled. least port backpressure enabled, then head-of-line blocking might occur.
1.3.3
Unicast Traffic (Subport Mode)
operate modes. first mode uses queueing model described previously, corresponds single channel approximately Gbit/s. second mode called subport mode. subport mode manage four channels Gbit/s each. physical interface Dataslice always single channel; subport mode four Gbit/s channels time multiplexed onto single Gbit/s channel. More precisely, given that ETT1 Chip always operates cells second, subport mode correspond four channels each 6.25M cells second. Figure shows block diagram ETT1 port card. seven) Dataslices provide single interface. separate device required manage four OC-48c channels merge them they appear single channel EPP/DS.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure ETT1 Port Operating Subport Mode with Four OC-48c Linecards
ETT1 Port board OC-48c Dataslice Dataslice Dataslice Dataslice Dataslice Dataslice cell/s interface OC-48c Scheduler Flow Control Crossbar Crossbar Crossbar Crossbar Crossbar Crossbar Crossbar
ETT1 Port board
OC-48c
OC-48c
support four best-effort priorities when operating subport mode. This primary feature that differentiates from Furthermore, operate mixed mode switch, ETT1 switch have some ports operating subport mode (quad OC-48c), while other ports operate normal mode (OC-192c). NOTE: been designed operate with same Dataslice device this certain limitations that designer should aware following sections subporting explain these limitations. Ports numbered through 31and each port EPP. Subports denoted through Ports with subports shown (port, subport) pair; example, port subport would shown (4,3) Ports without subports (OC-192c) simply numbered. following queueing description assumes ETT1 core configured with ports subport mode, supporting OC-48c linecards. First, consider single priority. Figure shows counters needed OC-48c linecard. There currently requests outstanding output (0,0). ETT1 Chip uses virtual output queues avoid head-of-line blocking issues, therefore, needs keep track unicast requests going each possible output channels (0,0) through (31,3). Consequently, counters needed. These counters reflect outstanding requests that this OC-48c ingress linecard made switch.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure Input Request Counters
request OC-48c linecard grant
31,3
support four OC-48c channels, each channel must have request counters each OC-48c output channels. Figure shows full counters single priority. Within Grant Manager process issues grants linecards response received requests. Once cell time Grant Manager selects request counter grant, given that many qualify. request counter qualifies receive grant counter non-zero there least empty cell buffer that used cells going that particular output. Once Grant Manager selected particular VOQ, decrements corresponding counter sends grant appropriate linecard. grant indicates specific within linecard. When linecard receives grant does things. First, sends cell body that currently head granted queue. Second, increments local credit counter that used determine whether request that queue sent.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure Full Counters Single Priority
request OC-48c linecard grant OC-48c
31,3 request OC-48c linecard grant 31,3 request OC-48c linecard grant 31,3 request OC-48c linecard grant OC-48c 31,3 OC-48c OC-48c
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
1.3.3.1
Virtual Output Queues
Ideally, EPP/DS would have separate which would correspond each request counters. This would mean that congestion would impact other same OC-48c VOQs different OC-48c channels. practice, constrained amount cell buffering available ETT1 Dataslice device (which contains actual cell buffers). Consequently, four request counters that correspond single output channel share single queue, shown Figure only manages single queue each output channels (whether output channel OC-48c OC-192c). connected four OC-48c inputs, then those four OC-48c linecards must share single queue used each output. four dotted arrows Figure show that four request counters output (0,0) same output (0,0).
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure Request Counters Single Output Share Single Queue
OC-48c linecard OC-48c OC-48c linecard OC-48c 31,0 OC-48c linecard OC-48c 31,0 31,1 OC-48c linecard OC-48c 31,3 31,3 31,2 (This cells going output OC-48c channel port Virtual Output Queues Dataslices) output channel)
31,3
31,3
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
complete sequence events input side shown Figure Linecard issues request send cell output channel (0,1). initial request count zero, arrival request then increments request count one. Grant Manager sees that there least empty cell buffer virtual output queue output (0,1), that request counter (0,1) non-zero decrements request counter (back zero) issues grant linecard. linecard then forwards actual cell body which stores cell virtual output queue. also issues request Scheduler, indicating that there cell waiting transferred output (0). Later Scheduler issues grant which forward cell through Crossbar output channel (the port
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure Sequence Events Input Side
OC-48c OC-48c linecard Scheduler Virtual Output Queues Crossbar
Initial state. Subport outstanding requests output (0,1)
OC-48c Request (0,1) OC-48c linecard Scheduler Virtual Output Queues Crossbar
Subport issues request send cell output (0,1). request count output (0,1) incremented. OC-48c OC-48c linecard Grant (0,1) Grant Manager Scheduler Virtual Output Queues Crossbar
Grant Manager issues grant queue (0,1) subport
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure (Continued)
OC-48c OC-48c linecard Scheduler Request Virtual Output Queues Crossbar
Linecard receives grant forwards cell body output (0,1). request made Scheduler.
OC-48c OC-48c linecard Scheduler Grant Virtual Output Queues Crossbar
Scheduler issues grant queue (0,1). then sends cell through crossbar output queue port
departure cell from will mean that there space least more cell that VOQ, that could trigger Grant Manager issue grant, assuming that request come that queue.
1.3.3.2
Output Process
Each consists input (i)EPP output (o)EPP. request counters virtual output queues described above reside iEPP. oEPP virtual input queues that receive cells forwarded Crossbar.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
ideal scenario, each oEPP would have separate virtual input queue every (possibly 128) input channel. However, same restriction applies, limiting oEPP just queues output OC-48c (and priority). Figure shows virtual input queues each oEPP when attached four OC-48c ports. Each virtual input queue maps back single virtual output queue iEPPs. abstract terms, virtual input queue simply extension appropriate virtual output queue. Each output OC-48c channel within oEPP unicast virtual input queues. output queue (virtual input queue) backpressured, then this will push back appropriate virtual output queue. However, each virtual output queue shared four input OC-48c channels within that iEPP, backpressuring virtual output queue effect asserting backpressure four OC-48c's same iEPP. oEPP Output Scheduler process which determines which cell should forwarded egress linecards.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure Virtual Input Queues
iEPP port Virtual Input Queues output OC-48c oEPP port
31,3
iEPP port
output OC-48c
31,3
output OC-48c
iEPP port
output OC-48c
31,3
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
1.3.3.3
Four Priorities
description preceding sections only considered single priority. supports four priorities. Therefore, there four times many counters queues have been described this only unicast traffic. four priorities modeled separate planes; input output queue figures shown earlier considered planes. four priorities then four copies these planes, stacked next each other third dimension. Figure illustrates this model reference counters virtual output queues. virtual input queues oEPP shown. Consequently, each iEPP 2048 request counters OC-48c input channels output channels priorities) virtual output queues (128 outputs priorities). Each oEPP virtual input queues output channels ports priorities). Remember, this calculation configuration ports OC-48c. Later describe number queues request counters differs some other possible configurations.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure Four Priorities
OC-48c OC-481c OC-48 OC-48c 31,3 OC-48c OC-481b OC-48 OC-48c 31,3 OC-48c OC-481b OC-48 OC-48c OC-48c OC-481b OC-481b OC-48c Priority Priority Priority Priority 31,0 31,1 31,2 31,3
1.3.3.4
Multiplexing Scheduler Requests Grants.
Figure Figure imply that ETT1 Scheduler accept requests from each ports where request specifies output channels. This case; Scheduler only maintains information port basis, subport basis.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Consequently, requests associated with cells going subports (0,0), (0,1), (0,2) (0,3) amalgamated become requests port NOTE: channel information lost; cell going channel (0,1) will still delivered channel port However, four queues have their requests multiplexed over single request/grant connection. Figure illustrates this concept.
Figure Scheduler Requests/Grants Multiplexed Port Level
VOQs iEPP cells output subport (0,0) cells output subport (0,1) Request send port cells output subport (0,2) Grant send port cells output subport (0,3) Scheduler
decisions make: which request issue, which four queues grant. cell time, iEPP have several possible requests which must communicate Scheduler. iEPP Scheduler Request Modulator which makes this decision. Section 3.1.3 "Scheduler Request Modulator" page discussion this algorithm. When iEPP receives grant from Scheduler must decide which four VOQs should serviced. simple round-robin algorithm used (across non-empty queues).
1.3.4
Multicast Traffic (subport mode)
Multicast cells handled port level, subport level. There single cell queue within iEPP, each priority multicast cells. Each OC-48c channel request counter, but, like unicasts, these four request counters single virtual output queue (per priority), shown Figure
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure Each iEPP Single Virtual Output Queue Multicast Cells
OC-48c
OC-48c
iEPP
Priority
OC-48c
OC-48c
multicast cells
OC-48c
OC-48c
OC-48c
OC-48c
before, this single queue limitation unless backpressure asserted queue, which case four OC-48c subports backpressured multicast cells from this port this priority. ETT1 uses multicast header identify multicast group that this cell should iEPP then maps into port vector with port. This port vector then passed Scheduler indicate list ports that should receive this multicast cell. Again, Scheduler only recognizes ports OC-48c subports; input side, multicast cell destined output ports (0,0), (0,1), (0,2) (0,3) would simply forwarded oEPP port oEPP receives multicast cell, together with tag. then maps (tag, source_port) into 4-bit vector that identifies what combination four subports should receive this multicast cell. Figure shows entire process.
Figure Multicast Cell Processing
iEPP multicast Crossbar Output Multicast Input Multicast Fanout Table Fanout Table multicast oEPP
Scheduler
Output Scheduler
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
order reduce blocking effects OC-48c channel over another channel same port, there special multicast output queue each priority. multicast cells forwarded through Crossbar still queued order they arrive. This special queue enables cells sent out-of-order, will always sent in-order given egress subport. Therefore, multicast cell waiting output queue memory only blocked cells destined same egress subport. Additional output queue memory becomes available whenever cell dequeued. backpressure asserted OC-48c channels output multicast queue, then subsequent cells going other channels same port blocked entire queue occupied cells destined blocked OC-48c channel. This consequence sharing output queue memory allocation multicast cells given priority. This cause multicast backpressure central Scheduler which block multicast cells destined other ports same priority multicast backpressure enabled.
1.3.4.1
Multicast Tables (OC-192c Quad OC-48c Modes)
input multicast fanout table (EITIBM EPP) always used multicast cells. Every EITIBM table with 4096 entries. uses 12-bit multicast field from label field determine which entry table should used each multicast cell, shown Figure Each entry simply vector where means that cell should that port. entry example, then multicast cell that uses that entry will sent port well other ports that have their
Figure EITIBM structure
iEPP Input Multicast Fanout Table EITIBM Multicast [11:0]
4095 Scheduler
each output OC-192c port
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
output multicast fanout table (EOTIBM EPP) only used port quad OC-48c mode. quad OC-48c mode multicast cell arrives oEPP crossbar. oEPP must determine what combination four OC-48c ports that this multicast cell should forwarded possible combinations assuming goes least ports). oEPP multicast bits) from label, that sufficient. sufficient because multicast tags managed independently across ports: each iEPP EITIBM given multicast might used specify different multicast fanouts different iEPPs. resolve ambiguity, oEPP must also include source port number with multicast tag, combines source port with multicast produce that unique across entire fabric. This then used index into table 4-bit entries shown Figure
Figure OITIBM structure
oEPP Output Multicast Fanout Table EOTIBM Multicast [11:0] Source port[4:0]
131071 Output Scheduler
each four local OC-48c ports
index combination {Multicast_tag, source_port}, that source port forms least significant bits address (index). Each entry four bits such that corresponds subport corresponds subport
1.3.5
Combinations OC-192c Quad OC-48c Linecards
switch might contain both OC-192c ports quad OC-48c ports. Each port card must configured with information about cards. number ingress queues function this ports: given quad OC-48c ports OC-192c ports, then each port will have unicast queues, where: priorities egress subports priorities then 512.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
port operating OC-192c mode then will also have unicast request counters. port operating quad OC-48c mode then will have unicast request counters separate request counters each OC-48c linecards. Each port always four multicast ingress queues. number egress queues function mode port itself, other ports: port OC-192c mode will have queue each input port priority, egress queues. port quad OC-48c mode will have four times many queues, queues. Each port always four multicast egress queues. size depth request counters queues defined Section "Input Dataslice Queue Memory Allocation with EPP" page Section "Output Dataslice Queue Memory Allocation with EPP" page 162.
1.3.6
Summary
Every ETT1 port number ingress request counters queues well egress queues. Given support ports, four subports port, four priorities, then there unicast ingress queues four multicast ingress queues, same number egress queues. every cell time, Scheduler arbitrates among ingress queues that eligible forward cell through Crossbar. ingress queue eligible non-empty destination egress queue backpressured. Scheduler makes decision using absolute priorities, with multicast requests having precedence over unicast requests same level, shown Figure
Figure Scheduler Priorities
Best Effort Traffic Priority Multicast Priority Unicast Priority Multicast Priority Unicast Priority Multicast Priority Unicast Priority Multicast Priority Unicast Lowest precedence Highest precedence
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
1.3.7
Protocol
this section, describe protocol from functional perspective explain works with queueing model that described previous section. NOTE: protocol defined "LCS Protocol Specification Protocol Version available from PMC-Sierra, Inc. This version supersedes Version Version first supported Chip with Enhanced Port Processor device (also referred ETT1 Chip Set) will supported future PMC-Sierra products. comparison versions protocol same document. primary role protocol credit-based flow control mechanism that ensures that cells transferred between linecard switch core only when sufficient buffer space available. order this, linecard must have knowledge queues that exist ETT1 core. protocol asymmetrical protocol; will first consider ingress direction. Figure shows best effort ingress queues ETT1 port (this assumes all-OC-192c switch). linecard have very different number queues (typically many more), thus must internal queues physical queues present ETT1 core.
Figure ETT1 Port Ingress Queues
Payload Header
Linecard
ETT1 Port
Cell requests payloads
Mapping unicast linecard ingress queues multicast multicast unicast Credit Information
example, linecard might provide Ainterface which manage many thousands virtual circuit flows, might have separate internal queue each flow. linecard must ingress queues ingress queues present ETT1 port. queues shown linecard have physically implemented provided that linecard perform requisite mapping from linecard queues ETT1 queues on-the-fly. ETT1 ingress queues Virtual Output Queues; mapping function simply each linecard queue desired egress port within ETT1 core. mapping function must also select appropriate priority more than single priority required.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure shows ingress cell requests data cells being forwarded ETT1 port. These cells fixed size format: cell consists eight byte header followed fixed length payload. payload bytes (depending number Dataslices Crossbars used), ports switch must same size payload. Chip never examines modifies cell payload unless cell used internal control purposes. linecard send cell request whenever cell arrives linecard linecard least request credit remaining appropriate queue. linecard must keep track number outstanding requests that send, must issue request unless corresponding credit. ETT1 port returns grant/credit information linecard. This grant/credit information reflects occupancy ingress queues ETT1 port. linecard only send cell given ingress queue within ETT1 port when receives grant from ETT1 port. linecard increments corresponding credit counter when receives grant/credit. order sustain maximum possible throughput, linecard must able send cell within short time receiving grant/credit from ETT1 port. linecard cannot this then ingress ETT1 port become empty which might, turn, affect throughput switch. system designer must aware time budget that really available linecard devices. grant/credit mechanism used egress direction. Rather, simpler Xon/Xoff-like mechanism used. oEPP will forward cells egress linecard whenever cells waiting output queues. linecard request send cell given priority, combination priorities, asserting hole request bits header. hole request asserted given priority, then request from linecard that should forward cell that priority cell time future. time between receiving hole request observing guaranteed more than cell times. Future compliant products will have different response times. Linecard designs requiring backpressure features should accommodate products which they will attach. linecard nearly exhausted egress buffers priority cells, then might continuously assert hole request priority until recover some additional buffers. NOTE: hole request mechanism operates per-priority per-source port. Refer "LCS Protocol Specification Protocol Version available from PMC-Sierra, Inc., further details protocol definition various fields used within header each direction.
EXPLICIT BACKPRESSURE FROM OEPP IEPP
iEPP does make unicast request Scheduler unless destination sufficient number empty buffers. each unicast egress there corresponding ingress VOQ. Every iEPP maintains state information occupancy relevant egress VIQs EPPs (including itself) order ensure that does forward cell full VIQ.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
actual occupancy dynamic: number free cell locations will increment whenever cell forwarded attached linecard, number free cell locations will decrement whenever cell enters from crossbar. iEPP knows latter information iEPP that forwards cell VIQ. iEPP does know when cell gets sent linecard. oEPP must tell each iEPP many cells have been sent from each relevant VIQ, this information must sent with sufficient frequency that throughput single flow adversely affected. ETT1 fabric, this update information transmitted from oEPPs iEPPs second Crossbar, referred Flow Control Crossbar. Figure shows cell flow from left right reverse flow update information.
Figure Separate Crossbar Used Convey Backpressure Information iEPPs
Virtual Output Queues
sync
Virtual Input Queues Cell Crossbar
Req/Gnt
Scheduler
iEPP
oEPP
Flow Control Crossbar
Every iEPP maintains counters (Output Queue Debit Counters) that track number cells each relevant VIQs every oEPP. reset, these counters zero. counter incremented whenever iEPP makes request forward cell appropriate VIQ. counter reaches pre-determined maximum, then iEPP will issue further requests. Once some cells forwarded from VIQ, iEPP will reduce counter resume issuing requests that flow. general, user does need aware details this mechanism except case when update information lost, perhaps noise-induced error more links attached Flow Control Crossbar. details this error recovery mechanism described Section "Enhanced Port Processor Flow Control Crossbar" page
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
1.4.1
Flow Control Crossbar Synchronization
Flow Control Crossbars interconnect every with every other EPP. During each cell time, Flow Control Crossbar sets crosspoints such that every iEPP receives update information from other oEPP. Further, iEPPs receive same update information from same oEPP. next cell time crosspoints must re-organized that every iEPP receives update information from different oEPP. system must ensure that Flow Control Crossbars EPPs synchronized, that every iEPP knows from which oEPP receiving information given time. This synchronization information generated Scheduler using FCCSYN register send periodic pulse EPPs. Further, redundant Scheduler used then Schedulers must synchronized. This Scheduler-to-Scheduler synchronization occurs after Scheduler Refresh operation. Refer Section 1.9.4 "Scheduler Refresh Procedure" page
SERVICE
service enables linecards periodically reserve bandwidth within ETT1 core. Cells that switched this service will have fixed latency through switch will delayed best effort contention output port. 1-in-N idle cells egress control packets cause temporary delays, these should absorbed guardband speedup. service implemented within ETT1 core using: dedicated queues ETT1 ports table-based reservation mechanism ETT1 ports flexible synchronization mechanism
1.5.1
Queues
linecard uses normal request-grant mechanism cells. When cells arrive ETT1 ingress port, they queued single queue which buffer cells. cells stored ingress queue until their reservation time occurs, which time they forwarded through crossbar appropriate egress queue. Given that there contention cells then queue might seem unnecessary. queue present that linecard does need closely synchronized with ETT1 fabric. port operating subport mode then four ingress queues used, each subport. Each ETT1 port single egress queue, even when port subport mode. ETT1 transmit cells out-of-order between subports, single shared queue does cause problem. Figure illustrates queueing structure.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure Queueing Structure
ETT1 Core
Linecard
ETT1 port
ETT1 port
Linecard
credits
Ingress queues
Figure also highlights some important points with respect service: linecard also require ingress egress queue. Unless arrival cells linecard synchronized with start Frame, linecard will need buffer those cells until appropriate time. There backpressure from ETT1 egress queue. Backpressure meaningful service within ETT1 core. egress linecard must accept cells that sent else discard them). normal credit mechanism used ingress cells.
1.5.2
Reservation Tables
service operates enabling ports reserve bandwidth through ETT1 crossbar specific times. Each port logically separate tables; sending receiving cells. Send Receive tables physically implemented single table called table. This table 1,024 entries, although fewer entries used Frame length less than 1024. Each entry form shown Table
Table Reservation Table
Field Input Input
Size (bits)
ETT1 Crossbar
Ingress queue
Egress queue
Egress queue
Meaning input side entry valid incoming cells (TDM flow
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Table
Reservation Table (Continued)
Field Subport Output Input Port Output vector
Size (bits)
Meaning Identifies source subport subport mode, else output side entry valid Input port that output expects receive from four-bit vector which defines fanout subports that should receive cell.
contains tables that table used fabric while system software configuring other table. EPPs must same table same time. Control Packet used specify which tables being used given time. start Frame, pointer each port begins entry table advances entry every cell time. During every cell time, "current" table Send entry valid, checks cell head appropriate queue flow (tag) that matches entry table. not, then nothing special done. they match then informs Scheduler that will sending cell some fixed time future. This prevents Scheduler from trying schedule best-effort cell from that port same time. Figure shows logic ingress port.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Figure Ingress Port Logic
Cell Queues
OC-48c
OC-48c
OC-48c
OC-48c
"Send"Table OC-48c Select Identifier
Current Slot Time
egress side port also looks table. Output Valid current entry set, then port tells Scheduler that expects receive cell some fixed time future. This prevents Scheduler from trying schedule best-effort cell that port same time. Scheduler doesn't receive input request from specified input port, then assumes that there cell ready transmitted, thus egress port that expected receive that cell will become available best-effort cell scheduling. When cell enters egress port, placed separate, 96-entry, egress queue dedicated traffic. Whenever this queue non-empty, cell head queue sent immediately linecard (provided that Output Scheduler frozen).
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
service inherently multicast service. egress ETT1 must therefore support ability cell combination four subports within port. four-bit field every table entry provides multicast fanout each cell that received egress port. keeps track this fanout information sends cell each subport that member fanout. Figure illustrates uses table determine source port cell, well local subport multicast fanout.
Figure oEPP Must Treat Transferred Cell Multicast Cell
Cell from Crossbar Output Cell Queue
0101
0111
0001
Scheduler
Input port
output vector 0101 (multicast fanout) Reservation Table
Current Slot Time
output cell queue does need large cells always forwarded output linecards highest priority, queue never backpressured. size this queue does, however, restrict possible table configuration. following constraints required prevent dropped cells: table time slot, input port send more than cell output port receive more than cell. Subport mode: Reservation Table specify same ingress subport only once every four time slots. Subport mode: Reservation Table specify same egress subport more than time slots within moving window time slots. This constraint caused output queue.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
1.5.3
Frame Timing
single table shown Figure addition 1,024 entry portion, there additional "guard band" entries preceding table. These entries unusable explained below. additional time slots (plus uncertainty/jitter sync timing) unusable following last valid entry tables pipeline latency through Scheduler EPP. These entries cannot used traffic.
Figure Single Table
TDM_Sync Guard Band TDM_Period Main Frame Unusable entries uncertainty TDM_Sync
Entries (128 cells)
Entries 1024
time between Sync signals referred period. This period plus length table plus final latency allowance uncertainty. chapter, referred "TDM offset value".
1.5.4
Synchronization
There several different levels synchronization required service work properly. First, EPPs must point same entry their tables order Send Receive reservations match Scheduler. Second, each linecard must synchronized (approximately) with ETT1 port that send cells right time third, that linecard will switch tables synchronously with ETT1 port. first level synchronization, between ETT1 ports, handled automatically ETT1 Chip Set. ETT1 devices operate cell synchronously. second third levels synchronization, between linecard ETT1, handled Sync mechanism. Scheduler configured three ways, described below, periodically send TDM_Sync ports. ETT1 ports receive TDM_Sync signal from Scheduler same cell time. Upon receiving TDM_Sync from Scheduler, every port then flushes cells currently input queue, transmits Control Packet linecard starts counter which increments every cell time. When this counter reaches value offset (guardband), begins comparing cell head FIFO with entry table location normal processing commences. TDM_Sync Control Packet both include Frame Select tell ports linecards which table use. This allows ports linecards switch tables synchronously.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
TDM_Sync signals arrive linecards exactly same cell time. There several cell times between first linecard receiving TDM_Sync last. variation variations link lengths also timing uncertainties when signals cross asynchronous clock domains. guard band absorbs these variations, linecard does need aware link delays clock domain issues. soon linecard receives TDM_Sync signal starts forward requests ETT1 port. port ingress queue these cells buffer them until they required. ingress queue cells deep, controlled normal request-grant flow-control mechanism. linecard must forward cells correct order (corresponding entries Send table), does need closely synchronized ETT1 core. linecard skip cells (i.e. send them), must re-order cells relative tables. ETT1 core waits cell times (the offset) after sent TDM_Sync before Frame started. This provides linecards with sufficient time receive TDM_Sync signal request send first cells. Figure shows sequence events more detail.
Figure Synchronization
Last usable slot previous Frame
TDM_Sync arrives EPPs
cells arrive ETT1 ports from linecards
slots +uncertainty unusable
TDM_Sync arrives linecards different times.
Frame starts
Scheduler configured send TDM_Sync three ways. Each driven these three sources: Suggested_TDM_Sync from counter EPP. counter Scheduler. Suggested_TDM_Sync from designated linecard (through EPP). There pros cons each these methods. Each method described more detail below.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
1.5.4.1
Sync Driven
programmed send Suggested_TDM_Sync Scheduler once every celltimes writing EPP's Sync Generation Control register writing Enable Sync Generation register. table select also field Sync Generation Control register. Scheduler programmed listen Suggested_TDM_Sync from designated port. advantage this method over third method that only exposure jitter (uncertainties delay) TDM_Sync after sends Control Packet linecard, crossing asynchronous clock domains (and subport-ordering delay subport mode). advantage this method over second method that when using Schedulers fault tolerance, second method requires synchronous writes program Schedulers exactly same time; this method provides synchronous Suggested_TDM_Sync Schedulers instead. possible disadvantage this method that Sync synchronous ETT1's core clock, linecard's clock other external synchronization signal.
1.5.4.2
Sync Driven Scheduler
Scheduler programmed generate TDM_Sync signals once every celltimes using Scheduler's Control register. table select also controlled Control register. Scheduler ignores Suggested_TDM_Sync from ports. advantages this method simplicity jitter, similar first method. disadvantage this method that when using Schedulers fault tolerance, this method requires synchronous writes program Schedulers exactly same time; Schedulers' Control registers enabled send Sync different times, then EPPs will report mismatching frames from Schedulers. Scheduler refresh will synchronize Scheduler-generated Syncs.
1.5.4.3
Sync Driven Designated Linecard
linecard send Suggested_TDM_Sync ETT1 port, using Control Packet, every TDM_Period cells. Control Packet also specifies which tables should used. When receives Control Packet sends Suggested_TDM_Sync signal Scheduler. Scheduler programmed listen Suggested_TDM_Sync from designated port. advantage this method that external synchronization source used. disadvantage this method that there much more exposure jitter TDM_Sync. Asynchronous clock domain crossings (and subport-ordering delays subport mode) will cause delay variations multiple stages: request Suggested_TDM_Sync Control Packet, grant, delivery Suggested_TDM_Sync Control Packet payload, finally delivery TDM_Sync Control Packet from ETT1 linecards. Additionally, Scheduler's internal digital TDM_Sync
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
(described below) adds jitter. "TDM Service ETT1 Switch more thorough discussion TDM_Sync jitter consequences.
Cores" Note
receiving Suggested_TDM_Sync signal, Scheduler computes period which Suggested_TDM_Sync signals arriving. Scheduler then uses this period determine next period which sends TDM_Sync ETT1 ports. period TDM_Sync signals will track period incoming Suggested_TDM_Sync signals. However phase TDM_Sync signals shifted relative Suggested_TDM_Sync signals. Internally Scheduler digital which will gradually cause transmitted TDM_Sync signal have fixed phase offset relative incoming Suggested_TDM_Sync signal. Figure shows passage signals.
Figure Signal Flow
ETT1 core Linecard Control Packet Suggested_TDM_Sync Designated Linecard Control Packet Control Packet ETT1 Port TDM_Sync ETT1 Scheduler ETT1 Port TDM_Sync
Linecard Control Packet
ETT1 Port
TDM_Sync
After count TDM_Period cell times, designated line card will send another Suggested_TDM_Sync, line card period counter will re-initialize process will repeat. Scheduler sees three periods without receiving another Suggested sync, will trigger interrupt continue send TDM_Syncs ports current period.
1.5.5
Configuring Service
Before cells flow system must configured. This involves following: Configure tables ports that will send receive cells. Configure Scheduler appropriate mode synchronization. Enable cells.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Configuring tables straightforward, described Section 1.5.2 "TDM Reservation Tables". rules tables are: port source more than cell cell time. port receive more than cell cell time. linecard must send cells EPP: order, before current slot reaches slot reserved that cell. Subport mode: Reservation Table specify same ingress subport only once every four time slots. Subport mode: Reservation Table specify same egress subport more than time slots within moving window time slots. This constraint caused output queue. Item above important understand: will buffer ingress cells before they sent through ETT1 fabric. only buffer cells ingress. linecard must able send cells meet average burst rate that implied table configuration. linecard operating cells/sec, example, while ETT1 fabric operates cells/sec then linecard must reserve contiguous burst slots because would unable supply last cells time their reservations. next step configure Scheduler. Scheduler service controlled Control register. first aspect configure source Suggested_TDM_sync signal. linecard will supply Suggested_TDM_sync signal then configure Control register with selected port Current Port valid bit. linecard does supply Suggested_TDM_ sync signal then Scheduler must configured with desired TDM_Period: write desired period into Initial Period field toggle Load Initial Period control bit. cases Enable (bit Scheduler's Control register should final stage configuring Scheduler write desired value Enable traffic register. Each bits should those ports that will send receive traffic.
1.5.6
Changing Source Suggested Sync
Scheduler configured external (port linecard) source Suggested_TDM_sync, then change source port writing value into port field Control register. Once port starts supply Suggested_TDM_sync signal then Scheduler will adjust frequency phase transmitted TDM_Sync signal match that incoming signal. Scheduler will change frequency outgoing sync signal once seen three Suggested_TDM_ sync signals, will only gradually adjust phase. phase outgoing TDM_Sync will adjusted cell time every TDM_period. NOTE: phase reference Scheduler received Suggested_TDM_sync signal,
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
which will several cell times after linecard sent Suggested_TDM_sync ETT1 port. Scheduler does incoming Suggested_TDM_sync signal three TDM_periods then will issue interrupt indicating that sync source failed. However will continue operate with current values frequency phase.
ETT1 USAGE PROTOCOL
This section provides details header used ETT1 system. Further information operation protocol "LCS Protocol Specification Protocol Version available from
1.6.1
Protocol Prepend
segments bytes length. line switch format shown Table while Table shows field definitions this format. Likewise, Tables show switch line format field definitions, respectively.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Table
Byte Valid Request Field
Format from Linecard Switch
Label_1 [13:7]
Byte
Label_1 [6:0]
Type_1 Rsvd
Byte Payload Valid Payload Field
Type_1 [2:0] Seq_Number[9:3]
Byte
Byte Byte Byte
Seq_Number[2:0] Payload_Hole_Request[3:0] CRC-16[15:8]
Rsvd Rsvd
Hole Field
Field Byte Bytes 8-71 8-83 CRC-16 [7:0] Payload Data
Payload
Table
Field Valid Label_1 Type_1 Payload Valid Seq_Number Payload_Hole_ Request Size (bits)
Definitions Fields from Linecard Switch
Definition Indicates that request field valid (Request Label_1). Label that being requested (e.g. Segment type that being requested (e.g. Control Packet). Indicates that Payload Data field contains valid payload. sequence number synchronizing grants with payloads. Request "hole" payload stream from switch linecard. (Priority 3,2,1,0 bits [3:0], respectively.) (See Section 1.2.4 "End-to-End Flow Control" page explanation). 16-bit calculated over complete 64-bit prepend.a Payload Data
CRC-16 Payload Data
bytes
Section 1.6.4.1 "Use CRC-16 Field" page details calculation Section 3.4.2.2 "Reset Control" page 169,
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Table
Byte Grnt Valid
Format from Switch Linecard
Label_1 [13:7]
Byte Grant Field Byte Type_1 [2:0]
Label_1 [6:0]
Type_1 Seq_Num [9:5] Payload Valid Label_2 [13:12]
Byte
Seq_Num [4:0]
Byte Payload Field Byte Byte Field Byte Bytes 8-71 8-83 Label_2 [3:0]
Label_2 [11:4] Type_2 [3:0] CRC-16 [15:8] CRC-16 [7:0] Payload Data
Payload
Table
Field Grnt Valid Label_1 Type_1 Seq_Num Payload Valid Label_2 Type_2 CRC-16 Payload Data Size (bits) bytes
Definitions Fields from Switch Linecard
Definition Indicates that grant field valid (Grant Label_1 Sequence Number). Label credit (e.g. QID/mcast-label/TDM-label). Segment type credit (e.g. TDM). Sequence Number associated with grant. Indicates whether egress payload valid (Payload Label_2 Payload Data). Label egress payload (e.g. ingress port identifier). Segment type egress payload (e.g. Control Packet). 16-bit calculated over complete 64-bit prepend.a Payload Data
Section 1.6.4.1 "Use CRC-16 Field" page details calculation Section 3.4.2.2 "Reset Control" page 169,
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
1.6.2
Label Type Fields
This section provides detail about generic 14-bit labels 4-bit types defined Table Table Table Table cases, multi-bit fields, transmitted first. each segment, "Type Field" "Label Field" encoded follows:
Table Encoding 4-bit Type Fields
Type[3:0] 0000 Segment Type Control Packet Provides inband control information between linecard switch core. Currently defined use. service provides preallocated bandwidth regular time interval. Prioritized unicast service where priority highest priority. Prioritized multicast service where priority highest priority.
0001,.,0110 0111
Reserved future
1000,.,1011
Unicast Best-Effort (Priority 0,1,2,3)
1100,.,1111
Multicast Best-Effort (Priority 0,1,2,3)
Table
Segment Type Control Packet Type_1 0000
Usage Ingress Request Label_1
Encoding Request Label_1[13:0] Rsvda (13), CP-category (1). CP-category Control Packets; CP-category Control Packets. (2)b, Expansionc (2), (10) (2)b, Expansiond (5), Destination Port (5), Destination subporte (2)b, Multicast (12)
Unicast Best-Effort (Priority 0,1,2,3) Multicast Best-Effort (Priority 0,1,2,3)
0111 10xx
11xx
Reserved. bits labeled Rsvd must zero. MUX[1:0] field placed beginning Request Label_1 field segments (i.e. both data traffic control packets). this field used function, used additional bits expansion. used, must zero. Expansion field used products number slots increased. this field used, "TDM Tag" field will increase contiguously. used, must zero. Unicast Expansion field used products number ports increased. this field used, "Destination Port" field will increase contiguously. used, must zero. port subports (i.e. connected single linecard without multiplexing), this field must zero.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Table
Segment Type Control Packet Type_1 0000
Usage Egress Grant Label_1
Encoding Grant Label_1[13:0] Rsvda (13), CP-category (1). CP-category Control Packets; CP-category Control Packets. (2)b, Expansionc (2), (10) (2)b, Expansiond (5), Destination Port (5), Destination subporte (2)b, Multicast (12)
Unicast Best-Effort (Priority 0,1,2,3) Multicast Best-Effort (Priority 0,1,2,3)
0111 10xx
11xx
Reserved. bits labeled Rsvd must zero. MUX[1:0] field placed beginning Grant Label_1 field segments (i.e. both data traffic control packets). this field used function, used additional bits expansion. used, must zero. Expansion field used products number slots increased. this field used, "TDM Tag" field will increase contiguously. used, must zero. Unicast Expansion field used products number ports increased. this field used, "Destination Port" field will increase contiguously. used, must zero. port subports (i.e. connected single linecard without multiplexing), this field must zero.
Table
Segment Type Control Packet Type_2 0000
Usage Egress Payload Label_2
Encoding Payload Label_2[13:0] Rsvda (13), CP-category (1). CP-category Control Packets; CP-category Control Packets. Expansionb (7), Source Port (5), Source subportc Expansiond (7), Source Port (5), Source subportc Expansione (7), Source Port (5), Source subportc
Unicast Best-Effort (Priority 0,1,2,3) Multicast Best-Effort (Priority 0,1,2,3)
0111 10xx
11xx
Reserved. bits labeled Rsvd must zero. Expansion field used products number ports increased. this field used, "Source Port" field will increase contiguously. used, must zero. source port subports (i.e. connected single linecard without multiplexing), this field will zero. Unicast Expansion field used products number ports increased. this field used, "Source Port" field will increase contiguously. used, must zero. Multicast Expansion field used products number ports increased. this field used, "Source Port" field will increase contiguously. used, must zero.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
1.6.2.1
Subport Mode Label Fields
Label fields arranged such that port addresses subdivided into four multiplexed, subport addresses. Each segment type with subport field utilizes bits [1:0] subport addresses. When subports utilized, these bits will zero. Subport multiplexing shares common physical connection carry segments to/from each subported linecard switch port. time multiplexing between subports done quad linecard, done intermediate stage. When time-multiplexed, segment transmissions sequenced between each four subports through field. When port device uses time-multiplexed subport mode, bits Request Label_1 field ingress Grant Label_1 field egress segments used multiplex/demultiplex each four subport channels. 2-bit field called MUX[1:0] field. ingress segments, MUX[1:0] bits carried Request Label_1 field; egress segments, MUX[1:0] bits carried Grant Label_1 (but Payload Label_2). Note that ingress traffic, this field used designate source subport ingress segment independent Request Label_1 subport field. Likewise egress traffic, field used designate destination subport egress segment independent Grant Label_1 Payload Label_2 subport fields. Segments entering switch core must have MUX[1:0] field correctly set. MUX[1:0] field arriving segments must follow sequence 00,01,10,11,00,. consecutive segments. MUX[1:0] field departing segments must also follow sequence 00,01,10,11,00,. consecutive segments. When subport sequence must interrupted need send Idle frame, sequence should continue with next subport sequence following previously sent subport. MUX[1:0] field placed beginning Request Label_1 Grant Label_1 field segments (i.e. both data traffic control packets). Segments departing arriving subported line card have MUX[1:0] field correspond linecard's designated subport. Optionally, linecard MUX[1:0] field zero, relying intermediary multiplexer device correctly MUX[1:0] field. such configuration, prepend generated checked masking MUX[1:0] field zero, enabling intermediary multiplexer device independently MUX[1:0] field without termination regeneration field. Note that without requirement setting MUX[1:0] field, linecard's framing function does have subport aware system with intermediary, time-multiplexed data stream.
1.6.3
Control Packets
link controlled Control Packets (CPs) that generated linecard switch core. When switch core sends linecard, marks correct Payload Label_2 Type_2 fields Control Packet sends control information payload data. When linecard wishes send switch core, goes through normal three phase request/grant/transmit process. Control Packets have higher priority over other traffic; therefore, grants will returned response request prior other outstanding requests. There three categories CPs: Control Packets that used perform low-level functions such link-synchronization, Control Packets which used communication between
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
linecard switch core CPU, Single Phase Control Packets which used special form Control packets timing sensitive information.
1.6.3.1
Control Packets
specified setting Label_1/Label_2 Field 14'h0000 (CP-category Control Packets) associated Type Field 4'h0 (Type Control Packets). Table describes contents first 10-bytes segment payload format.
Table
Field CP_Type CP_Type Extension Size (bits)
Control Packet Format
Definition Indicates type Control Packet (e.g. TDM)
Indicates information carried with this type control packet. Unused bits must zero. 16-bit calculated over 10-byte control packet payload.a
CRC-16
Section 1.6.4.1 "Use CRC-16 Field" page details calculation.
Because there flow control mechanism sent switch fabric, request/grant does distinguish CP_Type, care must taken overflow receiving queues. most, only request each CP_Type should outstanding. what follows, three CP_Types currently defined described. NOTE: actual headers Switch-to-Linecard Control Packets built EPP. Instead, expects find headers, preconfigured specific queue locations within Dataslices. Thus, part initialization sequence ETT1 port card write values Control Packets into Dataslice queues. Section "Output Dataslice Queue Memory Allocation with EPP" page 162, provides details these locations.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Control Packets format described Table
Table
Field CP_Type Frame Unused CRC-16 Size (bits)
Control Packet Format
Definition 00000000 Indicates which frame use. zero. 16-bit calculated over 10-byte control packet payload.
Note that when sent from linecard switch core, reply generated switch core. Request Count Control Packets Request Count only sent from linecard switch, used keep request counter values consistent between linecard switch core. When switch core receives this updates corresponding request counter (specified Label_1 Type_1) with count field. This control packet sent periodically when linecard suspects that linecard switch core have different values (e.g. following error). Request Count required unicast traffic also required multicast requested traffic when supported. unicast traffic, Label_1 field described Table multicast traffic, Label_1 field undefined should zero. format Request Count described Table
Table Request Count Control Packet Format
Field CP_Type Label_1 Type_1 Count Unused CRC-16 Size (bits) Definition 00000001 Label request counter. 10xx unicast 11xx multicast Update counter value, always zero multicast. zero. 16-bit calculated over 10-byte control packet payload.
PROPRIETARY CONFIDENTIAL PMC-SIERRA, INC., CUSTOMERS' INTERNAL
Released Data Sheet PMC-2000164 ISSUE
PM9311/2/3/5 ETT1CHIP
ENHANCED TT1SWITCH FABRIC
Start/Stop Control Packets Start/Stop Control Packets used toggle flow data traffic over link (here, data traffic defined TDM, best-effort unicast multicast segment payloads associated requests grants. i.e. non-Control Packet segments). Start/Stop Control Packets sent over link either direction. linecard sends Stop Control Packet switch core, switch core will stop sending grants linecard ingress data traffic, will stop sending egress data traffic linecard. When Start Control Packet sent, grants egress data traffic resume. switch core sends Stop Control Packet linecard, linecard must stop sending data traffic requests into switch core until Start Control Packet sent. Note that when either received Stop Control Packet, continue request, grant, send receive categories Control Packets. When link powers-up, will Stop mode: i.e. will Stop Control Packet been sent both directions.
Table
Field CP_Type Start/Stop Unused CRC-16 Size (bits)
Start/Stop Control Packet Format
Definition 00000010 1=Start, 0=Stop. zero. 16-bit calculated over 10-byte control packet payload.
1.6.3.2
Control Packets
Control Packets specified setting Label_1/Label_2 Field 14'h0001 (CP-category Control Packets) associated Type Field 4'h0 (Type Control Packets). When linecard sends switch core CPU, complete payload data passed switch core CPU. Likewise, when switch core sends linecard, complete payload data passed linecard. flow control mechanism explicitly implemented CPs, will implementation dependen
Other recent searches
NDR315 - NDR315 NDR315 Datasheet
MVL-313GH - MVL-313GH MVL-313GH Datasheet
isT-1 - isT-1 isT-1 Datasheet
HYM5V64404A - HYM5V64404A HYM5V64404A Datasheet
HYM5V64404AQ - HYM5V64404AQ HYM5V64404AQ Datasheet
HV632 - HV632 HV632 Datasheet
HT0740 - HT0740 HT0740 Datasheet
BLF378 - BLF378 BLF378 Datasheet
BCE0008G - BCE0008G BCE0008G Datasheet
AN3025 - AN3025 AN3025 Datasheet
TS4982 - TS4982 TS4982 Datasheet
Privacy Policy | Disclaimer
© 2012 Datasheet Archive
