Green Bus Example Implementation Specification

This is Robert's comment color.

Wolfgang likes this color.

Pavel likes this color.

Remarks

The content of the following page is still evolving. Here we try to concretise the basic ideas and to illustrate how the implementation could look like.

In ongoing experiments different possibilities of locating (slavebase/router/...) and creating (static/dynamic) threads are analysed. Furthermore we are trying to define a maximum number of required threads, which is sufficient to all conceivable bus architectures.

The results of those experiments will have a major impact on the final specification and therefore on the content of this page, too.

Architectural Aspects

The reference implementation specification is based on the OSCI TLM modeling architecture (figure 1).

Figure 1 Green Bus High Level Architecture

All green shaded components are planned to be generated automatically from an XML description. In the first example implementation, however, they will be implemented manually. From this, the XML schemas and XML generators will be developed subsequently.

In GreenBus-based system models, transactions are always initialized by a master which therefore calls one of its convenience API methods (in the example implementation this might be the TAC API). The number of method invocations for one transaction depends on the level of abstraction of the initiator port. For the PV level, usually one read() or write() call will be sufficient, while in the BA level, several transaction phases like request and data handshake phase will be reflected by several method calls.

To illustrate the ease of use of such a convenience API, the following code snippet shows how a master could look like using a high level API.

In the underlying GreenBus transport layer, transactions are split into several transfer phases (transfer atoms). Up to now we believe that the phases INIT, DATA-HANDSHAKE and FINALIZE may be sufficient. To this end, all convenience API calls (like read() or write()) are translated into the following calls:

Every master port contains the three request member objects init_req, data_hs_req and finalize_req, which are constructed and initialized during the port's constrcution. By that repeated request construction and destruction is avoided. The responses in the master port are simply response pointers. The used requests are all derived from the gs_request_base class and the responses are all derived from gs_response_base. Those base classes contain members that are common to all conceivable requests and responses (right now this is the Master ID [the ID is an tlm_tag], target address, rNw-flag and Req_type for requests and the resp_type for responses)

Router and Bus Protocol Details

The most significant modules of the GreenBus are the router module and the bus protocol module.

The router can be implemented in two ways:

• with multiple threads: an arbiter thread and one thread for each connected slave(reffered to as request handler threads). This can be achieved by dynamically creating the threads after elaboration. To keep control of eventual time-outs each slave base contains a thread, that executes the API method calls. In case of a long or even infinite wait in the implementation of the API method, the slave base thread will stall, but the request handler thread is still "alive" and can report such a stall.
• with a single arbiter thread:
  • the high simulation performance.
  • the implicit timing.
  • accumulated delay strategy by the use of the newly defined gs_wait statement. In this strategy, SystemC wait is executed at the end of the request elaboration.
  • SystemC wait() function calls are not allowed in methods of the bus protocol class.
  • : The router implementation depends on the user choice and a selected abstraction layer (PV, PVT, BA, CC).

The multiple thread implementation provides explicit timing for the bus protocol, resulting in more events and context switches compared to implicit timing model implemented by the single thread solution. Figure 2 and Figure 3 show a more detailed view of those two modules.

Multiple thread implementation (explicit timing)

Figure 2 Router with multiple threads implementation

In the router, there exists an ok_to_get event for each connected master and port and one thread for each connected slave.

The specification of the router and bus protocol gets clear with the help of an example:

When a master calls its send API method, the API sepcific init_request object (derived from gs_request_base) is configured with the appropriate information (target address, access_mode...), and then put to the router. The router's put(request) implementation enqueues the given request into the router's request queue and triggers the router internal do_arbitrate event. Due to this event the arbiter calls the get_request() method of the bus_protocol class. This call returns the next valid request from the request queue and the port rank of the slave connected to the initiator port of the router to which this request will be sent (both depend on the bus protocol). Then the arbiter checks the port rank and uses this one to forward the returned request to the corresponding request handler thread and starts this thread by triggering the corresponding event. Now the arbiter thread calls get_request again (there could be an request that can be handled concurrently to the other request). If this calls returns a NULL pointer (or something similar) the arbiter thread waits for the next do_arbitrate event. The request handler thread that was triggered, checks the request type (init, data_hs or finalize) and performs the according callbacks to the bus_protocol class. The callback functions implement the bus grant and addressing delays, bus state updates and eventual timeout control etc. These calls lead to call put(init_req) to the addressed slave base. If the slave responds within the timeout period by calling put(init_resp), the request handler thread triggers the corresponding ok_to_get event (the correct reference to this event was given to the master through its ok_to_get call and the given master ID) and the arbitrate event, since a new address may be transferred now. Now the master proceeds to the data-handshake phase, which equals the one described above. By implementing the get_request method, the bus state struct and all other callbacks correctly, maybe every bus can be moddeled.

You can find some MSC-like diagrams at the bottom of the router and bus protocol section of this page.

Single thread implementation (implicit timing)

Figure 3 Router with a single thread implementation

The arbiter thread plays a key role in the router implementation. The arbiter calls the get_request() method of the bus_protocol class. This method returns the next valid request from the priority queue. Then the arbiter determines the start time, the latency of the request and generates delay if it is necessary. The minimum initiation interval between successive requests is guaranteed. See Figure 4 for details. Then the arbiter does the same as request handler at the first router implementation. The bus protocol module can not use SystemC wait function but gs_wait instead. The gs_wait function accumulate delays which will be used by the arbiter at the end of the request elaboration. The gs_wait delays could be also generated from XML specification.

Figure 4 Router with a single thread timing diagram

The example implementation should model the behaviour of the IBM CoreConnect PLB.

Implementation Details (Proposal)

This is a very very draft spec, it is still based onto our request queue concept

Request and Response Objects

Since the infrastructure makes use of request and response objects (see above) the first thing to do is implementing those objects.

At first the gs_request_base and the gs_response_base must be implemented. They are template classes (at least the request base, where the address type is a template parameter), so that template specialisation can be used. The members of this bases where already mentioned above.

Before saying something to the plb-specific request and response objects, some words concerning the data to be transferred: To keep the example implementation effort as small as possible, we should transfer arrays (or vectors) of 64 bit data. Since the tac API (and that's the one we want to use) has a template data parameter we should set this to sc_unint<64>. The address template parameter should be set to unsigned long (32 bit).

After this the plb-specific requests and responses are to be implemented. The requests are derived from gs_request_base, the responses are derived from gs_response_base.

The plb-init_req must contain:

• a BE value,
• a Read/Write flag, (this one resides in the base)
• a BusLock flag*,
• a compressed flag**,
• a guarded flag**,
• a ordered flag**,
• a priority value,
• a burst flag,
• a size value*,
• a type value*,
• a lockErr flag**,
• a target address (this one resides in the base)

The members marked with one or two stars will certainly not be used in the first versions of the implementation. The members marked with two stars are only information for the slave and are not used by the bus.

The plb data_hs_req must contain:

• A data field (64 bit)
• A lastXfer flag

A finalize_req is not required.

The plb init_resp must contain:

• An integer encoding the acknowledgement type (this one may reside in the response_base), the actual meanings of the values depend on the bus
• Proposal: 1 equals primary ack, 2 equals secondary ack, 3 equals timeout

The plb data_resp must contain:

• A data field
• A word location integer. (only valid with line reads; this integer reflects the PLB's rdWdAddr qualifier)

The plb finalize_resp is not required.

The slave-terminates-burst, the variable-burst-length and the master-and-slave-differ-in-word-sizes features of the plb should be ignored in the first implementation.

This is also true for the slave-signals-error feature.

Having finsihed these objects the initiator port and slave base should be implemented:

Initiator Port and Slave Base

The initiator port translates the API into the bus atoms.

For example an high level API write is translated as follows (please refer to the following code as pseudo code, explicit casts, template stuff and so on are ignored):

A write_block(...) (or burst_write) looks the same, but the data_hs processing is put into a for-loop which repeats the data_hs method sequence until all data is transferred (lastXfer is set to true when the last data_hs transfer is set. Otherwise it's 0) Furthermore, in the init-phase the burst-flag is set to true, the size value is set to 0xB and the BE value doesn't matter any more.

The slave_base does the opposite of the initiator ports. It reconstructs the API calls out of the incoming puts and gets. (Therefore no between-phase delays can be added by the slave. This is only possible, when the slave is at a lower level of abstraction and does not use such a high level API any more.) The put implementation of the slave base simply stores the incoming request, resets the time-out flag and starts the slave base internal thread. The slavebase must have members that can store all API related information (like data, address, BE, rNw...)

• ok_to_get() returns a reference to the m_ok_to_get_event
• get() simply returns m_data_hs_resp and sets m_canGet to false
• nb_can_get() returns m_canGet

The slave-base thread takes care of the requests and does the needed API calls. In this way (when a API method stalls) the slave base thread stalls, too. But the request handler thread inside the router stays "alive" and can keep in control of time outs.

This kind of implementation only works correctly, when a slave can only be addressed by one master at a time. In case multiple masters access the slave the m_req object and all other transfer related members must guarded, so that they can only be overwritten when the active transfer phase is finished. This could be achieved by using a semaphor, that gets locked when put is called for the first time and is unlocked when the thread has finished processing the active phase. Whenever there's a put call with the semaphor beeing locked, then put will return without doing anything (besides returning false...).

We still have to work out a better way of supporting the one-slave-gets-accessed-by-multiple-masters-simultaneously feature.

Router and Bus Protocol

• The router contains two arrays of events. One ok_to_get_event and one thread_start_event for each thread. The size of the ok_to_get_event array is equal to the connected master ports. The size of the thread_start_event array is equal to the slaves connected to the initiator port of the Router.
• The idea is that the port rank returned by the getRequest method of the bus protocol module corresponds to the slave which will recieve the current request. The port rank defines which thread handles the current request and furthermore the call event_array[portRank] should return the event that is associated with this port rank.
• The router is a template class where the template parameter T defines the type of the address.
• The router has an sc_port<bus_protocol_if<T> > let's name it bfm_port. The template parameter T is the type of address used by the protocol.
• The router contains a request priority queue (rq), that stores and orders all incoming requests.
• The router contains a response vector (rspv), that contains the responses (or pointers to responses). The size of the rspv is equal to the connected masters.
• The router contains an active_request vector (arv), which stores the active request of each thread. The size of the arv is equal to the connected slaves.

The router implements the tlm_blocking_put and the tlm_get_if. This means the methods put(), nb_get(), nb_can_get, ok_to_get() and get() must be implemented. Not all of them are used.

The implementation of the put() method simply enqueues the request and notifies the arbiter, that some arbitration has to be done.

The ok_to_get and get methods make use of a tlm_tag. This tag is the masterID and ensures that the master gets his event and data.

The arbiter is a thread sensitive to a do_arbitrate event, as long as the bus_protocol can retrieve valid requests from the request queue, the arbiter thread "passes" them to the handler threads and notifies the according start_events:

A request handler thread is sensitive to its thread start event; it's generic for all busses so it has to cover all phases, even those which are not part of the plb. The names of the callback functions (e.g. WriteToTarget) differ from those in figure 2. The number and names of the callbacks have not been fixed, yet. After reviewing more contributions and analysing more busses, we will come up with a final generic set of callback functions.

The request handler simply checks the type of its active request and then performs the callback sequence for this request type.

The router implements the whole put-get-if once more but with a "do_" in front. These methods will be called by the bus protocol callback functions and invoke the transfers to the targets. (They can be referred to as callback-callbacks).

The router is the generic part, the bus specific behaviour (in this case the PLB) is implemented in the bus_protocol class.

The bus_protocol class contains no threads, it just implements the bus_protocol_if, a bus_state struct and maybe some helper methods. The bus_state_member may be called state;

The following enums may be of use when modelling the PLB.

The next pseudo code snippet shows a struct that may be able to reflect all of the PLB's states.

For example: When ass is IDLE, rdss is BUSY, wrss is IDLE , active_prim_rd_master and active_sec_rd_master and active_prim_wr_master are not equal to 255 and active_sec_wr_master is equal to 255.

Now the bus state is:

One master (active_prim_rd_master) has finsihed its address phase and is now reading data (rdss is BUSY).

Two other masters (active_sec_rd_master and active_prim_wr_master) have also finished their address phases (ass IDLE) and this implies that active_sec_rd_master is now waiting to become the primary read master and that active_prim_wr_master is just advancing to the write data stage (wss is still IDLE).

Now an incoming init read request would be stalled (there are already a primary and a secondary read master) an incoming init write request would be processed (it would become a secondary write reuqest). The only data-hs-requests that can be accepted, are data-hs-requests from active_prim_rd_master and active_prim_wr_master.

The following pseudo code snippet shows an example of a Callback function: InitXferToTarget.

The function would have only been called, when there when the address stage is IDLE and there is a free read or write slot (primary or secondary). Therefore the function does not have to check wether there is an available slot or not. It simply checks which one is free.

At first the function checks wether it deals with a write or a read request. Assuming that the request is a read request, now the value of active_prim_rd_master gets checked. If this equals the "no-active-master" value (in our case 255) the primary slot is free and so the function decodes the address (this is protocol specific and therefore also a part of the bus protocol class).

Then the active_prim_rd_master value is set to the new master, the address stage state is set to busy and the ADDRESS_DELAY wait is executed. This delay depends wether the PLB uses 1,2 or even 3 cycle arbitration.

After this the request is forwarded to the target by calling do_put and then the function waits for the ok_to_get event and the address timeout period. After the wait is over the function checks wether the wait was released by the ok_to_get event (do_get returns true) or by the timeout (do_get returns false).

Finally the address stage state is set to IDLE. After this the master can now put data-hs-requests into the request queue, which will be returned by getRequest() since this master is now the active_prim_rd_master.

In this way all acesses to the callbacks update the busstate, check eventual timeouts and so on. The most critical function will be the get_request method. There the whole request queue must be checked, wether there are new init_requests or data_hs_requests that can be processed. Every call has to update the busstate. For example a call to WriteToTarget has to update the state of the data stage.

Following, you'll find three links to some diagrams. The diagrams are MSC related but aren't UML standard compliant. They try to illustrate the flow of method calls and events using the GreenBus with the tac API. All bus protocol specific waits and bus state updates have been left out.

Concerning the Diagrams: Please ignore the request-handler-thread destruction. this is not correct. The request-handler-thread is NOT destructed after it has finished processing the request!

Request Queue

The request queue is a priority queue that stores and orders all incoming requests. Figure 5 shows a set of priority queue classes.

Figure 5 Priority queuers: Inheritance diagram

The ordering of the queue elements is controlled by CmpT<T> class.

Therefore it is easy to change the sort schema without any modification in the priority queue. The router uses the pure software priority queue for the PV, PVT layers and the adopted variant based on the sc_prim_channel for BA, CC layers (See Figure 6).

Figure 6 SWPqueue, PQueue are used by the Router

The SW priority queue (SWPQueue):

• automatically ordering of the elements in the queue by the priority.
• ordering is performed by external class CmpT<T>: easy to change/replace.
• provides a high performance: 10% efficient compared to STL priority_queue.
• based on the STL algorithms: make_heap, push_heap, pop_heap: operation speed in the worst case is O(log(N)).
• provides a more flexible and useful user interface.
• provides a different element storage policy of the queue: FIXED or VARIABLE.
• is a template class and implements a base abstract queue interface PQueueIface:
  • bool IsEmpty()
  • bool IsFull()
  • unsigned int Size()
  • unsigned int Capacity()
  • storagePolicy Policy()
  • const T* Top()
  • bool Push(const T& x)
  • void Pop()
  • void Clear()
  • void Recreate(const unsigned int & capacity,const storagePolicy & policy)

The HW priority queue (PQueue):

• adopts the functionality of the SWPQueue to the HW area.
• provides a high simulation performance: is s primitive channel with evaluation/update schema.
• implements the OSCI TLM unidirectional tlm_put_if and tlm_get_peek_if interfaces.
• can be used in blocking and non blocking parts of the code.