
EXPERIMENTAL feature: shared interrupt threads

Introduction
------------

The shared interrupt threads patch is an EXPERIMENT to investigate sharing
interrupt threads for more than one IRQ handler.  Sharing interrupt threads
is exected to:

   - reduce memory usage
   - reduce the number of tasks switches required to handle interrupts
     occuring closely together


Usage
-----

The RT PREEMPT real-time patch set modifies request_irq() so that creates
a kernel thread for each interrupt for which the IRQF_NODELAY flag is not
specified.  When the interrupt thread wakes it calls do_hardirq(), which
calls each of the actions registered for the IRQ.  The kernel thread then
calls schedule(), potentially sleeping until the next interrupt arrives.
When an interrupt arrives, the kernel interrupt handling code wakes the
appropriate interrupt thread.

The shared interrupt thread feature makes the following changes:

  1) Adds new functions to specify that the interrupt thread will be
     shared.

     setup_irq_shared_thread(unsigned int irq, struct irqaction *act,
        int shared_thread_num)

        setup_irq_shared_thread() is called the same way as setup_irq().

        If shared_thread is zero then this function behaves the same as
        setup_irq().

        shared_thread_num requests which shared thread to assign the IRQ to.
        The value of shared_thread_num may be in the range 0..9.  (This value
        is defined by NUM_SHARED_THREAD and may be changed in future
        implementations.)

     request_irq_shared_thread(unsigned int irq, irq_handler_t handler,
        unsigned long irqflags, const char *devname, void *dev_id,
        int shared_thread, int shared_thread_num)

        request_irq_shared_thread() is called the same way as request_irq().

        shared_thread_num requests which shared thread to assign the IRQ to.
        The value of shared_thread_num may be in the range 0..9.  (This value
        is defined by NUM_SHARED_THREAD and may be changed in future
        implementations.)

  2) A new config option forces all non-shared threaded IRQ interrupt handlers
     into shared interrupt thread zero.  This allows testing of shared
     interrupt threads with no other changes to the kernel or drivers.  If
     this configuration option is selected, it does not override the
     shared_thread_num that was selected by setup_irq_shared_thread() or
     request_irq_shared_thread().

     SNSC_FORCE_SHARED_IRQ_THREAD
     "Force non-shared IRQ threads into shared IRQ thread zero"
            Threaded interrupts can be placed into a shared IRQ thread instead
            of one thread per interrupt by calling setup_irq_shared_thread()
            instead of setup_irq().  This config option forces all IRQs
            created by setup_irq() to be placed into shared IRQ thread zero
            instead of separate threads per interrupt.


There are several reasons to have more than one shared interrupt thread:

  1) Allow different groups of interrupt handlers to be assigned
     different priorities.

  2) Allow different groups of interrupt handlers to be assigned
     different cpu affinity.

  3) Reduce the overhead of scanning through the list of interrupt handlers
     serviced by a shared interrupt thread.

     * Consider grouping interrupt sources that occur together in a shared
       interrupt thread.

     * Consider segregating rarely occuring interrupt sources in a shared
       interrupt thread.


TODO
----

The maximum number of shared interrupt threads is defined by the constant
NUM_SHARED_THREAD in kernel/irq/manage.c.  The current value is an arbitrary
choice that should work for the target application this experiment was
created for.  For a more generic solution, the value of this constant
should be tunable, perhaps by a CONFIG option.


Limitations
-----------

There is no check that all the interrupts serviced by a shared
interrupt thread are on the same cpu.  This simplifies the shared_thread
patch, but makes administration more difficult.

Setting the cpu affinity of an IRQ may alter the programming of the interrupt
controller hardware to set interrupt controller cpu affinity at the hardware
level.  For systems where this is the case, the interrupt controller cpu
affinity of each IRQ source that share the same shared interrupt thread
is independent.

The affinity of the IRQ that created the shared insterrupt thread will be
honored for the shared thread.  This means that if the cpu affinity of the
IRQ that created the shared interrupt thread is changed then the cpu affinity
of the shared interrupt thread will also change.  The IRQ that created the
shared interrupt thread is the first IRQ for which request_irq() or
request_irq_shared_thread() was called (after ok_to_create_irq_threads is set
to 1).  This may not be deterministic.  It is recommended that the cpu affinity
of all of the IRQs that share a thread be set to the same value.

Interrupt threads are allowed to sleep.  The author of an threaded interrupt
handler may choose to use this capability.  If a shared interrupt thread
sleeps while servicing such an interrupt then all other pending interrupts
handled by the thread will be blocked until the thread wakes.  This means
that normally you should use a shared interrupt thread for an interrupt
handler that may sleep.
