Discussion:
[dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding
(too old to reply)
Tomasz Kulasek
2015-06-03 10:58:59 UTC
Permalink
OVERVIEW
--------
1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding
device fully RSS-capable, so all slaves are synchronized with its configuration.
This mode is intended to provide RSS configuration as known from "dynamic RSS
configuration for one port" and made slaves transparent for client application
implementation.

2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not
synchronized. That provides an ability to configure them manually. This mode may
be useful when application wants to manage RSS in an unusual way and the
consistency of RSS configuration for slaves isn't required.

Turning on/off RSS mode for slaves when bonding is started is not possible.
Other RSS configuration is propagated over slaves, when bonding device API is
used to do it.

Tomasz Kulasek (8):
bond: dynamic rss configuration
ring: dynamic rss configuration
test: dynamic rss configuration
bond: queue stats mapping
ring: queue stats mapping set dummy implementation
examples: dynamic rss configuration for bonding
doc: fixed spellings and typos
doc: dynamic rss configuration for bonding

app/test/Makefile | 1 +
app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 40 +-
drivers/net/bonding/rte_eth_bond_api.c | 22 +
drivers/net/bonding/rte_eth_bond_pmd.c | 222 ++++-
drivers/net/bonding/rte_eth_bond_private.h | 11 +
drivers/net/ring/rte_eth_ring.c | 133 ++-
examples/bond_rss/Makefile | 59 ++
examples/bond_rss/bondrss.c | 293 +++++++
examples/bond_rss/bondrss.h | 163 ++++
examples/bond_rss/config.c | 251 ++++++
examples/bond_rss/ui.c | 915 ++++++++++++++++++++
12 files changed, 2759 insertions(+), 25 deletions(-)
create mode 100644 app/test/test_link_bonding_rssconf.c
create mode 100644 examples/bond_rss/Makefile
create mode 100644 examples/bond_rss/bondrss.c
create mode 100644 examples/bond_rss/bondrss.h
create mode 100644 examples/bond_rss/config.c
create mode 100644 examples/bond_rss/ui.c
--
1.7.9.5
Tomasz Kulasek
2015-06-03 10:59:00 UTC
Permalink
Bonding device implements independent management of RSS settings. It stores its
own copies of settings i.e. RETA, RSS hash function and RSS key. It’s required
to ensure consistency.

1) RSS hash function set for bonding device is maximal set of RSS hash functions
supported by all bonded devices. That mean, to have RSS support for bonding, all
slaves should be RSS-capable.

2) RSS key is propagated over the slaves "as is", and is always 40 bytes long.

3) RETA for bonding is an internal table managed by bonding API, and is used as
a pattern to set up slaves. Its size is GCD of all RETA sizes, so it can be
easily used as a pattern providing expected behavior, even if slaves RETA sizes
are different.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/bonding/rte_eth_bond_api.c | 22 ++++
drivers/net/bonding/rte_eth_bond_pmd.c | 192 ++++++++++++++++++++++++++--
drivers/net/bonding/rte_eth_bond_private.h | 11 ++
3 files changed, 211 insertions(+), 14 deletions(-)

diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c
index e91a623..a31b098 100644
--- a/drivers/net/bonding/rte_eth_bond_api.c
+++ b/drivers/net/bonding/rte_eth_bond_api.c
@@ -301,6 +301,7 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id)
internals->active_slave_count = 0;
internals->rx_offload_capa = 0;
internals->tx_offload_capa = 0;
+ internals->flow_type_rss_offloads = 0;

memset(internals->active_slaves, 0, sizeof(internals->active_slaves));
memset(internals->slaves, 0, sizeof(internals->slaves));
@@ -366,6 +367,11 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
memset(&dev_info, 0, sizeof(dev_info));
rte_eth_dev_info_get(slave_port_id, &dev_info);

+ /* We need to store slaves reta_size to be able to synchronize RETA for all
+ * slave devices even if its sizes are different.
+ */
+ internals->slaves[internals->slave_count].reta_size = dev_info.reta_size;
+
if (internals->slave_count < 1) {
/* if MAC is not user defined then use MAC of first slave add to
* bonded device */
@@ -379,9 +385,16 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
/* Make primary slave */
internals->primary_port = slave_port_id;

+ /* Inherit queues settings from first slave */
+ internals->nb_rx_queues = slave_eth_dev->data->nb_rx_queues;
+ internals->nb_tx_queues = slave_eth_dev->data->nb_tx_queues;
+
+ internals->reta_size = dev_info.reta_size;
+
/* Take the first dev's offload capabilities */
internals->rx_offload_capa = dev_info.rx_offload_capa;
internals->tx_offload_capa = dev_info.tx_offload_capa;
+ internals->flow_type_rss_offloads = dev_info.flow_type_rss_offloads;

} else {
/* Check slave link properties are supported if props are set,
@@ -400,6 +413,13 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
}
internals->rx_offload_capa &= dev_info.rx_offload_capa;
internals->tx_offload_capa &= dev_info.tx_offload_capa;
+ internals->flow_type_rss_offloads &= dev_info.flow_type_rss_offloads;
+
+ /* RETA size is GCD of all slaves RETA sizes,
+ * so, if all sizes will be the power of 2, the lower one is GCD */
+ if (internals->reta_size > dev_info.reta_size)
+ internals->reta_size = dev_info.reta_size;
+
}

internals->slave_count++;
@@ -528,6 +548,8 @@ __eth_bond_slave_remove_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
if (internals->slave_count == 0) {
internals->rx_offload_capa = 0;
internals->tx_offload_capa = 0;
+ internals->flow_type_rss_offloads = 0;
+ internals->reta_size = 0;
}
return 0;
}
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index c937e6b..4f6fc68 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -1306,6 +1306,15 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev,
if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC)
slave_eth_dev->data->dev_conf.intr_conf.lsc = 1;

+ /* If RSS is enabled for bonding, try to enable it for slaves */
+ if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) {
+ memcpy(
+ &(slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf),
+ &(bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf),
+ sizeof(bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf));
+ slave_eth_dev->data->dev_conf.rxmode.mq_mode |= ETH_MQ_RX_RSS;
+ }
+
/* Configure device */
errval = rte_eth_dev_configure(slave_eth_dev->data->port_id,
bonded_eth_dev->data->nb_rx_queues,
@@ -1349,6 +1358,29 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev,
}
}

+ /* If RSS is enabled for bonding, synchronize RETA */
+ if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) {
+ int i;
+ struct bond_dev_private *internals;
+ internals = bonded_eth_dev->data->dev_private;
+
+ for (i = 0; i < internals->slave_count; i++) {
+ if (internals->slaves[i].port_id == slave_eth_dev->data->port_id) {
+ errval = rte_eth_dev_rss_reta_update(
+ slave_eth_dev->data->port_id,
+ &(internals->reta_conf[0]),
+ internals->slaves[i].reta_size);
+ if (errval != 0) {
+ RTE_BOND_LOG(ERR,
+ "rte_eth_dev_rss_reta_update: port=%d, err (%d)",
+ slave_eth_dev->data->port_id, errval);
+ return -1;
+ }
+ break;
+ }
+ }
+ }
+
/* Start device */
errval = rte_eth_dev_start(slave_eth_dev->data->port_id);
if (errval != 0) {
@@ -1569,6 +1601,9 @@ bond_ethdev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)

dev_info->rx_offload_capa = internals->rx_offload_capa;
dev_info->tx_offload_capa = internals->tx_offload_capa;
+ dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads;
+
+ dev_info->reta_size = internals->reta_size;
}

static int
@@ -1954,21 +1989,126 @@ bond_ethdev_lsc_event_callback(uint8_t port_id, enum rte_eth_event_type type,
}
}

+static int
+bond_ethdev_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf,
+ uint16_t reta_size)
+{
+ int i, j, result = 0;
+ int slave_reta_size;
+ int reta_count;
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ if (reta_size != internals->reta_size)
+ return -EINVAL;
+
+ /* Copy RETA table */
+ reta_count = reta_size / RTE_RETA_GROUP_SIZE;
+
+ for (i = 0; i < reta_count; i++) {
+ internals->reta_conf[i].mask = reta_conf[i].mask;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ internals->reta_conf[i].reta[j] = reta_conf[i].reta[j];
+ }
+
+ /* Fill rest of array (512 - max slaves RETA size) */
+ for (; i < 512 / RTE_RETA_GROUP_SIZE; i += reta_count)
+ memcpy(&(internals->reta_conf[i]), &(internals->reta_conf[0]),
+ sizeof(internals->reta_conf[0]) * reta_count);
+
+ /* Propagate RETA over slaves */
+ for (i = 0; i < internals->slave_count; i++) {
+ slave_reta_size = internals->slaves[i].reta_size;
+ result = rte_eth_dev_rss_reta_update(internals->slaves[i].port_id,
+ &(internals->reta_conf[0]), slave_reta_size);
+ if (result < 0)
+ return result;
+ }
+
+ return 0;
+}
+
+static int
+bond_ethdev_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf,
+ uint16_t reta_size)
+{
+ int i, j;
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ if (reta_size != internals->reta_size)
+ return -EINVAL;
+
+ /* Copy RETA table */
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++)
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ reta_conf[i].reta[j] = internals->reta_conf[i].reta[j];
+
+ return 0;
+}
+
+static int
+bond_ethdev_rss_hash_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ int i, result = 0;
+ struct bond_dev_private *internals = dev->data->dev_private;
+ struct rte_eth_rss_conf bond_rss_conf;
+
+ memcpy(&bond_rss_conf, rss_conf, sizeof(struct rte_eth_rss_conf));
+
+ bond_rss_conf.rss_hf &= internals->flow_type_rss_offloads;
+
+ if (bond_rss_conf.rss_hf != 0)
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = bond_rss_conf.rss_hf;
+
+ if (bond_rss_conf.rss_key != NULL)
+ memcpy(internals->rss_key, bond_rss_conf.rss_key, 40);
+
+ for (i = 0; i < internals->slave_count; i++) {
+ result = rte_eth_dev_rss_hash_update(internals->slaves[i].port_id,
+ &bond_rss_conf);
+ if (result < 0)
+ return result;
+ }
+
+ return 0;
+}
+
+static int
+bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ if (rss_conf->rss_key != NULL)
+ memcpy(rss_conf->rss_key, internals->rss_key, 40);
+
+ return 0;
+}
+
struct eth_dev_ops default_dev_ops = {
- .dev_start = bond_ethdev_start,
- .dev_stop = bond_ethdev_stop,
- .dev_close = bond_ethdev_close,
- .dev_configure = bond_ethdev_configure,
- .dev_infos_get = bond_ethdev_info,
- .rx_queue_setup = bond_ethdev_rx_queue_setup,
- .tx_queue_setup = bond_ethdev_tx_queue_setup,
- .rx_queue_release = bond_ethdev_rx_queue_release,
- .tx_queue_release = bond_ethdev_tx_queue_release,
- .link_update = bond_ethdev_link_update,
- .stats_get = bond_ethdev_stats_get,
- .stats_reset = bond_ethdev_stats_reset,
- .promiscuous_enable = bond_ethdev_promiscuous_enable,
- .promiscuous_disable = bond_ethdev_promiscuous_disable
+ .dev_start = bond_ethdev_start,
+ .dev_stop = bond_ethdev_stop,
+ .dev_close = bond_ethdev_close,
+ .dev_configure = bond_ethdev_configure,
+ .dev_infos_get = bond_ethdev_info,
+ .rx_queue_setup = bond_ethdev_rx_queue_setup,
+ .tx_queue_setup = bond_ethdev_tx_queue_setup,
+ .rx_queue_release = bond_ethdev_rx_queue_release,
+ .tx_queue_release = bond_ethdev_tx_queue_release,
+ .link_update = bond_ethdev_link_update,
+ .stats_get = bond_ethdev_stats_get,
+ .stats_reset = bond_ethdev_stats_reset,
+ .promiscuous_enable = bond_ethdev_promiscuous_enable,
+ .promiscuous_disable = bond_ethdev_promiscuous_disable,
+ .reta_update = bond_ethdev_rss_reta_update,
+ .reta_query = bond_ethdev_rss_reta_query,
+ .rss_hash_update = bond_ethdev_rss_hash_update,
+ .rss_hash_conf_get = bond_ethdev_rss_hash_conf_get
};

static int
@@ -2049,6 +2189,30 @@ bond_ethdev_configure(struct rte_eth_dev *dev)
int arg_count;
uint8_t port_id = dev - rte_eth_devices;

+ static const uint8_t default_rss_key[40] = {
+ 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D,
+ 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+ 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B,
+ 0xBE, 0xAC, 0x01, 0xFA
+ };
+
+ int i, j;
+
+ /* If RSS is enabled, fill table and key with default values */
+ if (dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) {
+
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = internals->rss_key;
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 40;
+ memcpy(internals->rss_key, default_rss_key, 40);
+
+ for (i = 0; i < 512 / RTE_RETA_GROUP_SIZE; i++) {
+ internals->reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ internals->reta_conf[i].reta[j] = j % dev->data->nb_rx_queues;
+ }
+
+ }
+
/*
* if no kvlist, it means that this bonded device has been created
* through the bonding api.
diff --git a/drivers/net/bonding/rte_eth_bond_private.h b/drivers/net/bonding/rte_eth_bond_private.h
index 45e5c65..26f4b28 100644
--- a/drivers/net/bonding/rte_eth_bond_private.h
+++ b/drivers/net/bonding/rte_eth_bond_private.h
@@ -103,6 +103,8 @@ struct bond_slave_details {
uint8_t last_link_status;
/**< Port Id of slave eth_dev */
struct ether_addr persisted_mac_addr;
+
+ uint16_t reta_size;
};


@@ -155,6 +157,15 @@ struct bond_dev_private {
uint32_t rx_offload_capa; /** Rx offload capability */
uint32_t tx_offload_capa; /** Tx offload capability */

+ /** Bit mask of RSS offloads, the bit offset also means flow type */
+ uint64_t flow_type_rss_offloads;
+
+ uint16_t reta_size;
+ struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE];
+
+ uint8_t rss_key[40]; /**< 40-byte hash key. */
+ uint8_t rss_key_len; /**< hash key length in bytes. */
+
struct rte_kvargs *kvlist;
uint8_t slave_update_idx;
};
--
1.7.9.5
Tomasz Kulasek
2015-06-03 10:59:01 UTC
Permalink
This implementation allows to set and read RSS configuration for ring device,
and is used to validate right values propagation over the slaves, in test units
for dynamic RSS configuration for bonding.

It have no impact on packet processing by ring device.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/ring/rte_eth_ring.c | 122 +++++++++++++++++++++++++++++++++++++--
1 file changed, 118 insertions(+), 4 deletions(-)

diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index 6832f01..6d2182f 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -39,6 +39,7 @@
#include <rte_string_fns.h>
#include <rte_dev.h>
#include <rte_kvargs.h>
+#include <rte_spinlock.h>

#define ETH_RING_NUMA_NODE_ACTION_ARG "nodeaction"
#define ETH_RING_ACTION_CREATE "CREATE"
@@ -66,6 +67,17 @@ struct pmd_internals {
struct ring_queue tx_ring_queues[RTE_PMD_RING_MAX_TX_RINGS];

struct ether_addr address;
+
+ /** Bit mask of RSS offloads, the bit offset also means flow type */
+ uint64_t flow_type_rss_offloads;
+
+ rte_spinlock_t rss_lock;
+
+ uint16_t reta_size;
+ struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_128 /
+ RTE_RETA_GROUP_SIZE];
+
+ uint8_t rss_key[40]; /**< 40-byte hash key. */
};


@@ -173,6 +185,8 @@ eth_dev_info(struct rte_eth_dev *dev,
dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues;
dev_info->min_rx_bufsize = 0;
dev_info->pci_dev = NULL;
+ dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads;
+ dev_info->reta_size = ETH_RSS_RETA_SIZE_128;
}

static void
@@ -234,6 +248,93 @@ static int
eth_link_update(struct rte_eth_dev *dev __rte_unused,
int wait_to_complete __rte_unused) { return 0; }

+static int
+eth_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf,
+ uint16_t reta_size)
+{
+ int i, j;
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ if (reta_size != internal->reta_size)
+ return -EINVAL;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ /* Copy RETA table */
+ for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) {
+ internal->reta_conf[i].mask = reta_conf[i].mask;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ internal->reta_conf[i].reta[j] = reta_conf[i].reta[j];
+ }
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf,
+ uint16_t reta_size)
+{
+ int i, j;
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ if (reta_size != internal->reta_size) return -EINVAL;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ /* Copy RETA table */
+ for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) {
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ reta_conf[i].reta[j] = internal->reta_conf[i].reta[j];
+ }
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_hash_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0)
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ rss_conf->rss_hf & internal->flow_type_rss_offloads;
+
+ if (rss_conf->rss_key != NULL)
+ memcpy(internal->rss_key, rss_conf->rss_key, 40);
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ if (rss_conf->rss_key != NULL)
+ memcpy(rss_conf->rss_key, internal->rss_key, 40);
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
static const struct eth_dev_ops ops = {
.dev_start = eth_dev_start,
.dev_stop = eth_dev_stop,
@@ -250,6 +351,10 @@ static const struct eth_dev_ops ops = {
.stats_reset = eth_stats_reset,
.mac_addr_remove = eth_mac_addr_remove,
.mac_addr_add = eth_mac_addr_add,
+ .reta_update = eth_rss_reta_update,
+ .reta_query = eth_rss_reta_query,
+ .rss_hash_update = eth_rss_hash_update,
+ .rss_hash_conf_get = eth_rss_hash_conf_get
};

int
@@ -268,6 +373,13 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[],

unsigned i;

+ static const uint8_t default_rss_key[40] = {
+ 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D,
+ 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+ 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B,
+ 0xBE, 0xAC, 0x01, 0xFA
+ };
+
/* do some parameter checking */
if (rx_queues == NULL && nb_rx_queues > 0)
goto error;
@@ -316,12 +428,14 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[],

internals->nb_rx_queues = nb_rx_queues;
internals->nb_tx_queues = nb_tx_queues;
- for (i = 0; i < nb_rx_queues; i++) {
+ for (i = 0; i < nb_rx_queues; i++)
internals->rx_ring_queues[i].rng = rx_queues[i];
- }
- for (i = 0; i < nb_tx_queues; i++) {
+ for (i = 0; i < nb_tx_queues; i++)
internals->tx_ring_queues[i].rng = tx_queues[i];
- }
+ internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK;
+ internals->reta_size = ETH_RSS_RETA_SIZE_128;
+
+ memcpy(internals->rss_key, default_rss_key, 40);

eth_drv->pci_drv.name = ring_ethdev_driver_name;
eth_drv->pci_drv.id_table = id_table;
--
1.7.9.5
Tomasz Kulasek
2015-06-03 10:59:02 UTC
Permalink
This test module uses ring device to check right RSS configuration propagation.

1) Propagation test
a) Set RSS hash function for bonding, fetch RSS hash function from bonded
slave, check if same. Do it for all slaves.
b) Repeat above for RSS key and RETA.

2)Dynamic adding slave to the bonding
a) Remove slave from bonding.
b) Change its configuration to other than bonding.
c) Add slave to the started bonding device.
d) Check if slaves configuration changed.

3) Repeat point 1) and 2) with RSS multi-queue mode enabled/disabled for bonding
port.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
app/test/Makefile | 1 +
app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++++++++++++++++++++++
2 files changed, 675 insertions(+)
create mode 100644 app/test/test_link_bonding_rssconf.c

diff --git a/app/test/Makefile b/app/test/Makefile
index 3c777bf..aef9ae7 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -134,6 +134,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_ACL) += test_acl.c
ifeq ($(CONFIG_RTE_LIBRTE_PMD_RING),y)
SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_mode4.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_rssconf.c
endif

SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring.c
diff --git a/app/test/test_link_bonding_rssconf.c b/app/test/test_link_bonding_rssconf.c
new file mode 100644
index 0000000..c0be755
--- /dev/null
+++ b/app/test/test_link_bonding_rssconf.c
@@ -0,0 +1,674 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <errno.h>
+#include <rte_cycles.h>
+#include <sys/queue.h>
+
+#include <rte_byteorder.h>
+#include <rte_common.h>
+#include <rte_debug.h>
+#include <rte_ethdev.h>
+#include <rte_log.h>
+#include <rte_lcore.h>
+#include <rte_memory.h>
+
+#include <rte_string_fns.h>
+
+#include <rte_eth_ring.h>
+#include <rte_errno.h>
+#include <rte_eth_bond.h>
+
+#include "test.h"
+
+#define SLAVE_COUNT (4)
+
+#define RXTX_RING_SIZE 1024
+#define RXTX_QUEUE_COUNT 4
+
+#define BONDED_DEV_NAME ("rssconf_bond_dev")
+
+#define SLAVE_DEV_NAME_FMT ("rssconf_slave%d")
+#define SLAVE_RXTX_QUEUE_FMT ("rssconf_slave%d_q%d")
+
+#define NUM_MBUFS 8191
+#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)
+#define MBUF_CACHE_SIZE 250
+#define BURST_SIZE 32
+
+#define INVALID_SOCKET_ID (-1)
+#define INVALID_PORT_ID (0xFF)
+#define INVALID_BONDING_MODE (-1)
+
+struct slave_conf {
+ uint8_t port_id;
+ struct rte_eth_dev_info dev_info;
+
+ struct rte_eth_rss_conf rss_conf;
+ uint8_t rss_key[40];
+ struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE];
+
+ uint8_t is_slave;
+ struct rte_ring *rxtx_queue[RXTX_QUEUE_COUNT];
+};
+
+struct link_bonding_rssconf_unittest_params {
+ uint8_t bond_port_id;
+ struct rte_eth_dev_info bond_dev_info;
+ struct rte_eth_rss_reta_entry64 bond_reta_conf[512 / RTE_RETA_GROUP_SIZE];
+ struct slave_conf slave_ports[SLAVE_COUNT];
+
+ struct rte_mempool *mbuf_pool;
+};
+
+static struct link_bonding_rssconf_unittest_params test_params = {
+ .bond_port_id = INVALID_PORT_ID,
+ .slave_ports = {
+ [0 ... SLAVE_COUNT - 1] = { .port_id = INVALID_PORT_ID, .is_slave = 0}
+ },
+ .mbuf_pool = NULL,
+};
+
+/**
+ * Default port configuration with RSS turned off
+ */
+static struct rte_eth_conf default_pmd_conf = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_NONE,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled */
+ .hw_ip_checksum = 0, /**< IP checksum offload enabled */
+ .hw_vlan_filter = 0, /**< VLAN filtering disabled */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled */
+ .hw_strip_crc = 0, /**< CRC stripped by hardware */
+ },
+ .txmode = {
+ .mq_mode = ETH_MQ_TX_NONE,
+ },
+ .lpbk_mode = 0,
+};
+
+static struct rte_eth_conf rss_pmd_conf = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_RSS,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled */
+ .hw_ip_checksum = 0, /**< IP checksum offload enabled */
+ .hw_vlan_filter = 0, /**< VLAN filtering disabled */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled */
+ .hw_strip_crc = 0, /**< CRC stripped by hardware */
+ },
+ .txmode = {
+ .mq_mode = ETH_MQ_TX_NONE,
+ },
+ .rx_adv_conf = {
+ .rss_conf = {
+ .rss_key = NULL,
+ .rss_hf = ETH_RSS_IPV6,
+ },
+ },
+ .lpbk_mode = 0,
+};
+
+#define FOR_EACH(_i, _item, _array, _size) \
+ for (_i = 0, _item = &_array[0]; _i < _size && (_item = &_array[_i]); _i++)
+
+/* Macro for iterating over every port that can be used as a slave
+ * in this test.
+ * _i variable used as an index in test_params->slave_ports
+ * _slave pointer to &test_params->slave_ports[_idx]
+ */
+#define FOR_EACH_PORT(_i, _port) \
+ FOR_EACH(_i, _port, test_params.slave_ports, \
+ RTE_DIM(test_params.slave_ports))
+
+static int
+configure_ethdev(uint8_t port_id, struct rte_eth_conf *eth_conf, uint8_t start)
+{
+ int rxq, txq;
+
+ TEST_ASSERT(rte_eth_dev_configure(port_id, RXTX_QUEUE_COUNT,
+ RXTX_QUEUE_COUNT, eth_conf) == 0, "Failed to configure device %u",
+ port_id);
+
+ for (rxq = 0; rxq < RXTX_QUEUE_COUNT; rxq++) {
+ TEST_ASSERT(rte_eth_rx_queue_setup(port_id, rxq, RXTX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id), NULL,
+ test_params.mbuf_pool) == 0, "Failed to setup rx queue.");
+ }
+
+ for (txq = 0; txq < RXTX_QUEUE_COUNT; txq++) {
+ TEST_ASSERT(rte_eth_tx_queue_setup(port_id, txq, RXTX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id), NULL) == 0,
+ "Failed to setup tx queue.");
+ }
+
+ if (start) {
+ TEST_ASSERT(rte_eth_dev_start(port_id) == 0,
+ "Failed to start device (%d).", port_id);
+ }
+
+ return 0;
+}
+
+/**
+ * Remove all slaves from bonding
+ */
+static int
+remove_slaves(void)
+{
+ unsigned n;
+ struct slave_conf *port;
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ if (port->is_slave) {
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(
+ test_params.bond_port_id, port->port_id),
+ "Cannot remove slave %d from bonding", port->port_id);
+ port->is_slave = 0;
+ }
+ }
+
+ return 0;
+}
+
+static int
+remove_slaves_and_stop_bonded_device(void)
+{
+ TEST_ASSERT_SUCCESS(remove_slaves(), "Removing slaves");
+ rte_eth_dev_stop(test_params.bond_port_id);
+ return TEST_SUCCESS;
+}
+
+/**
+ * Add all slaves to bonding
+ */
+static int
+bond_slaves(void)
+{
+ unsigned n;
+ struct slave_conf *port;
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ if (!port->is_slave) {
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id,
+ port->port_id), "Cannot attach slave %d to the bonding",
+ port->port_id);
+ port->is_slave = 1;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * Set all RETA values in port_id to value
+ */
+static int
+reta_set(uint8_t port_id, uint8_t value, int reta_size)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ int i, j;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) {
+ /* select all fields to set */
+ reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ reta_conf[i].reta[j] = value;
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
+
+/**
+ * Check if slaves RETA is synchronized with bonding port. Returns 1 if slave
+ * port is synced with bonding port.
+ */
+static int
+reta_check_synced(struct slave_conf *port)
+{
+ unsigned i;
+
+ for (i = 0; i < test_params.bond_dev_info.reta_size;
+ i++) {
+
+ int index = i / RTE_RETA_GROUP_SIZE;
+ int shift = i % RTE_RETA_GROUP_SIZE;
+
+ if (port->reta_conf[index].reta[shift] !=
+ test_params.bond_reta_conf[index].reta[shift])
+ return 0;
+
+ }
+
+ return 1;
+}
+
+/**
+ * Fetch bonding ports RETA
+ */
+static int
+bond_reta_fetch(void) {
+ unsigned j;
+
+ for (j = 0; j < test_params.bond_dev_info.reta_size / RTE_RETA_GROUP_SIZE;
+ j++)
+ test_params.bond_reta_conf[j].mask = ~0LL;
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(test_params.bond_port_id,
+ test_params.bond_reta_conf, test_params.bond_dev_info.reta_size),
+ "Cannot take bonding ports RSS configuration");
+ return 0;
+}
+
+/**
+ * Fetch slaves RETA
+ */
+static int
+slave_reta_fetch(struct slave_conf *port) {
+ unsigned j;
+
+ for (j = 0; j < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; j++)
+ port->reta_conf[j].mask = ~0LL;
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(port->port_id,
+ port->reta_conf, port->dev_info.reta_size),
+ "Cannot take bonding ports RSS configuration");
+ return 0;
+}
+
+/**
+ * Remove and add slave to check if slaves configuration is synced with
+ * the bonding ports values after adding new slave.
+ */
+static int
+slave_remove_and_add(void)
+{
+ struct slave_conf *port = &(test_params.slave_ports[0]);
+
+ /* 1. Remove first slave from bonding */
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(test_params.bond_port_id,
+ port->port_id), "Cannot remove slave #d from bonding");
+
+ /* 2. Change removed (ex-)slave and bonding configuration to different
+ * values
+ */
+ reta_set(test_params.bond_port_id, 1, test_params.bond_dev_info.reta_size);
+ bond_reta_fetch();
+
+ reta_set(port->port_id, 2, port->dev_info.reta_size);
+ slave_reta_fetch(port);
+
+ TEST_ASSERT(reta_check_synced(port) == 0,
+ "Removed slave didn't should be synchronized with bonding port");
+
+ /* 3. Add (ex-)slave and check if configuration changed*/
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id,
+ port->port_id), "Cannot add slave");
+
+ bond_reta_fetch();
+ slave_reta_fetch(port);
+
+ return reta_check_synced(port);
+}
+
+/**
+ * Test configuration propagation over slaves.
+ */
+static int
+test_propagate(void)
+{
+ unsigned i;
+ uint8_t n;
+ struct slave_conf *port;
+ uint8_t bond_rss_key[40];
+ struct rte_eth_rss_conf bond_rss_conf;
+
+ int retval = 0;
+ uint64_t rss_hf = 0;
+ uint64_t default_rss_hf = 0;
+
+ rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info);
+
+ /*
+ * Test hash function propagation
+ */
+ for (i = 0; i < sizeof(test_params.bond_dev_info.flow_type_rss_offloads)*8;
+ i++) {
+
+ rss_hf = test_params.bond_dev_info.flow_type_rss_offloads & (1<<i);
+ if (rss_hf) {
+ bond_rss_conf.rss_key = NULL;
+ bond_rss_conf.rss_hf = rss_hf;
+
+ retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id,
+ &bond_rss_conf);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set slaves hash function");
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ retval = rte_eth_dev_rss_hash_conf_get(port->port_id,
+ &port->rss_conf);
+ TEST_ASSERT_SUCCESS(retval,
+ "Cannot take slaves RSS configuration");
+
+ TEST_ASSERT(port->rss_conf.rss_hf == rss_hf,
+ "Hash function not propagated for slave %d",
+ port->port_id);
+ }
+
+ default_rss_hf = rss_hf;
+ }
+
+ }
+
+ /*
+ * Test key propagation
+ */
+ for (i = 1; i < 10; i++) {
+
+ /* Set all keys to zero */
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ memset(port->rss_conf.rss_key, 0, 40);
+ retval = rte_eth_dev_rss_hash_update(port->port_id,
+ &port->rss_conf);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RSS keys");
+ }
+
+ memset(bond_rss_key, i, sizeof(bond_rss_key));
+ bond_rss_conf.rss_hf = default_rss_hf,
+ bond_rss_conf.rss_key = bond_rss_key;
+ bond_rss_conf.rss_key_len = 40;
+
+ retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id,
+ &bond_rss_conf);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set bonded port RSS keys");
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ retval = rte_eth_dev_rss_hash_conf_get(port->port_id,
+ &(port->rss_conf));
+
+ TEST_ASSERT_SUCCESS(retval,
+ "Cannot take slaves RSS configuration");
+
+ /* compare keys */
+ retval = memcmp(port->rss_conf.rss_key, bond_rss_key,
+ sizeof(bond_rss_key));
+ TEST_ASSERT(retval == 0, "Key value not propagated for slave %d",
+ port->port_id);
+ }
+ }
+
+ /*
+ * Test RETA propagation
+ */
+ for (i = 0; i < RXTX_QUEUE_COUNT; i++) {
+
+ /* Set all keys to zero */
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ retval = reta_set(port->port_id, (i + 1) % RXTX_QUEUE_COUNT,
+ port->dev_info.reta_size);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RETA");
+ }
+
+ TEST_ASSERT_SUCCESS(reta_set(test_params.bond_port_id,
+ i % RXTX_QUEUE_COUNT, test_params.bond_dev_info.reta_size),
+ "Cannot set bonded port RETA");
+
+ bond_reta_fetch();
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ slave_reta_fetch(port);
+ TEST_ASSERT(reta_check_synced(port) == 1, "RETAs inconsistent");
+ }
+ }
+
+ return TEST_SUCCESS;
+}
+
+/**
+ * Test propagation logic, when RX_RSS mq_mode is turned on for bonding port
+ */
+static int
+test_rss(void)
+{
+ /**
+ * Configure bonding port in RSS mq mode
+ */
+ TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id,
+ &rss_pmd_conf, 0), "Failed to configure bonding device\n");
+
+ rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info);
+
+ TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed");
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id),
+ "Failed to start bonding port (%d).", test_params.bond_port_id);
+
+ TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed");
+
+ TEST_ASSERT(slave_remove_and_add() == 1, "New slave should be synced");
+
+ remove_slaves_and_stop_bonded_device();
+
+ return TEST_SUCCESS;
+}
+
+/**
+ * Test propagation logic, when RX_RSS mq_mode is turned off for bonding port
+ */
+static int
+test_rss_lazy(void)
+{
+ TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id,
+ &default_pmd_conf, 0), "Failed to configure bonding device\n");
+
+ rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info);
+
+ TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed");
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id),
+ "Failed to start bonding port (%d).", test_params.bond_port_id);
+
+ TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed");
+
+ TEST_ASSERT(slave_remove_and_add() == 0, "New slave shouldn't be synced");
+
+ remove_slaves_and_stop_bonded_device();
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_setup(void)
+{
+ unsigned i, n;
+ int retval;
+ char name[256];
+ char rng_name[RTE_RING_NAMESIZE];
+ struct slave_conf *port;
+
+ if (test_params.mbuf_pool == NULL) {
+
+ test_params.mbuf_pool = rte_mempool_create("MBUF_POOL", NUM_MBUFS *
+ SLAVE_COUNT, MBUF_SIZE, MBUF_CACHE_SIZE,
+ sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init,
+ NULL, rte_pktmbuf_init, NULL, rte_socket_id(), 0);
+
+ TEST_ASSERT(test_params.mbuf_pool != NULL,
+ "rte_mempool_create failed\n");
+ }
+
+ /* Create / initialize ring eth devs. */
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ snprintf(name, sizeof(name), SLAVE_DEV_NAME_FMT, n);
+
+ for (i = 0; i < RXTX_QUEUE_COUNT; i++) {
+ snprintf(rng_name, sizeof(rng_name), SLAVE_RXTX_QUEUE_FMT, n, i);
+
+ if (port->rxtx_queue[i] == NULL) {
+ port->rxtx_queue[i] = rte_ring_create(rng_name, RXTX_RING_SIZE,
+ 0, RING_F_SP_ENQ|RING_F_SC_DEQ);
+ TEST_ASSERT(port->rxtx_queue[i] != NULL,
+ "Failed to allocate rx ring '%s': %s", name,
+ rte_strerror(rte_errno));
+ }
+ }
+
+ if (rte_eth_from_rings(name, port->rxtx_queue, RXTX_QUEUE_COUNT,
+ port->rxtx_queue, RXTX_QUEUE_COUNT, 0)) {
+ TEST_ASSERT(retval >= 0,
+ "Failed to create ring ethdev '%s'\n", name);
+ }
+
+ port->port_id = rte_eth_dev_count() - 1;
+ port->rss_conf.rss_key = port->rss_key;
+ port->rss_conf.rss_key_len = 40;
+
+ retval = configure_ethdev(port->port_id, &default_pmd_conf, 1);
+ TEST_ASSERT_SUCCESS(retval, "Failed to configure virtual ethdev %s\n",
+ name);
+
+ rte_eth_dev_info_get(port->port_id, &port->dev_info);
+ }
+
+ if (test_params.bond_port_id == INVALID_PORT_ID) {
+ retval = rte_eth_bond_create(BONDED_DEV_NAME, 0, rte_socket_id());
+
+ TEST_ASSERT(retval >= 0, "Failed to create bonded ethdev %s",
+ BONDED_DEV_NAME);
+
+ test_params.bond_port_id = retval;
+
+ TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id,
+ &default_pmd_conf, 0), "Failed to configure bonding device\n");
+
+ rte_eth_dev_info_get(test_params.bond_port_id,
+ &test_params.bond_dev_info);
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+check_environment(void)
+{
+ return TEST_SUCCESS;
+}
+
+static int
+test_rssconf_executor(int (*test_func)(void))
+{
+ int test_result;
+
+ /* Check if environment is clean. Fail to launch a test if there was
+ * a critical error before that prevented to reset environment. */
+ TEST_ASSERT_SUCCESS(check_environment(),
+ "Refusing to launch test in dirty environment.");
+
+ RTE_VERIFY(test_func != NULL);
+ test_result = (*test_func)();
+
+ /* If test succeed check if environment wast left in good condition. */
+ if (test_result == TEST_SUCCESS)
+ test_result = check_environment();
+
+ /* Reset environment in case test failed to do that. */
+ if (test_result != TEST_SUCCESS) {
+ TEST_ASSERT_SUCCESS(remove_slaves_and_stop_bonded_device(),
+ "Failed to stop bonded device");
+ }
+
+ return test_result;
+}
+
+static int
+test_setup_wrapper(void)
+{
+ return test_rssconf_executor(&test_setup);
+}
+
+static int
+test_rss_wrapper(void)
+{
+ return test_rssconf_executor(&test_rss);
+}
+
+static int
+test_rss_lazy_wrapper(void)
+{
+ return test_rssconf_executor(&test_rss_lazy);
+}
+
+static struct unit_test_suite link_bonding_rssconf_test_suite = {
+ .suite_name = "RSS Dynamic Configuration for Bonding Unit Test Suite",
+ .setup = test_setup,
+ .unit_test_cases = {
+ TEST_CASE_NAMED("test_setup", test_setup_wrapper),
+ TEST_CASE_NAMED("test_rss", test_rss_wrapper),
+ TEST_CASE_NAMED("test_rss_lazy", test_rss_lazy_wrapper),
+ { NULL, NULL, NULL, NULL, NULL } /**< NULL terminate unit test array */
+ }
+};
+
+static int
+test_link_bonding_rssconf(void)
+{
+ return unit_test_suite_runner(&link_bonding_rssconf_test_suite);
+}
+
+static struct test_command test_link_bonding_rssconf_cmd = {
+ .command = "link_bonding_rssconf_autotest",
+ .callback = test_link_bonding_rssconf,
+};
+
+REGISTER_TEST_COMMAND(test_link_bonding_rssconf_cmd);
--
1.7.9.5
Tomasz Kulasek
2015-06-03 10:59:03 UTC
Permalink
Queue stats mapping is used in example application. This patch adds propagation
of mapping over the slaves, and fills bonding port's stats with a sum of
corresponding values taken from bonded slaves, when stats are requested for
bonding port.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/bonding/rte_eth_bond_pmd.c | 30 +++++++++++++++++++++++++++++-
1 file changed, 29 insertions(+), 1 deletion(-)

diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 4f6fc68..7a4ecb8 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -1766,7 +1766,7 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
struct bond_dev_private *internals = dev->data->dev_private;
struct rte_eth_stats slave_stats;

- int i;
+ int i, j;

/* clear bonded stats before populating from slaves */
memset(stats, 0, sizeof(*stats));
@@ -1788,6 +1788,15 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
stats->rx_pause_xon += slave_stats.rx_pause_xon;
stats->tx_pause_xoff += slave_stats.tx_pause_xoff;
stats->rx_pause_xoff += slave_stats.rx_pause_xoff;
+
+ for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) {
+ stats->q_ipackets[j] += slave_stats.q_ipackets[j];
+ stats->q_opackets[j] += slave_stats.q_ipackets[j];
+ stats->q_ibytes[j] += slave_stats.q_ipackets[j];
+ stats->q_obytes[j] += slave_stats.q_ipackets[j];
+ stats->q_errors[j] += slave_stats.q_ipackets[j];
+ }
+
}
}

@@ -2090,6 +2099,24 @@ bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev,
return 0;
}

+static int
+bond_ethdev_queue_stats_mapping_set(struct rte_eth_dev *dev,
+ uint16_t queue_id, uint8_t stat_idx, uint8_t is_rx)
+{
+ int i;
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ for (i = 0; i < internals->slave_count; i++)
+ if (is_rx)
+ rte_eth_dev_set_rx_queue_stats_mapping (
+ internals->slaves[i].port_id, queue_id, stat_idx);
+ else
+ rte_eth_dev_set_tx_queue_stats_mapping (
+ internals->slaves[i].port_id, queue_id, stat_idx);
+
+ return 0;
+}
+
struct eth_dev_ops default_dev_ops = {
.dev_start = bond_ethdev_start,
.dev_stop = bond_ethdev_stop,
@@ -2103,6 +2130,7 @@ struct eth_dev_ops default_dev_ops = {
.link_update = bond_ethdev_link_update,
.stats_get = bond_ethdev_stats_get,
.stats_reset = bond_ethdev_stats_reset,
+ .queue_stats_mapping_set = bond_ethdev_queue_stats_mapping_set,
.promiscuous_enable = bond_ethdev_promiscuous_enable,
.promiscuous_disable = bond_ethdev_promiscuous_disable,
.reta_update = bond_ethdev_rss_reta_update,
--
1.7.9.5
Tomasz Kulasek
2015-06-03 10:59:04 UTC
Permalink
Per queue statistics are already implemented for ring device, but with static
mapping (stat_idx == queue_id).

This fix is required, if you want to use ring device in test application and is
used only to point that per queue statistics are provided for this device.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/ring/rte_eth_ring.c | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index 6d2182f..07695c3 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -335,6 +335,16 @@ eth_rss_hash_conf_get(struct rte_eth_dev *dev,
return 0;
}

+static int
+eth_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id,
+ __rte_unused uint8_t stat_idx,
+ __rte_unused uint8_t is_rx)
+{
+ /* Do nothing, just return ok */
+ return 0;
+}
+
static const struct eth_dev_ops ops = {
.dev_start = eth_dev_start,
.dev_stop = eth_dev_stop,
@@ -347,6 +357,7 @@ static const struct eth_dev_ops ops = {
.rx_queue_release = eth_queue_release,
.tx_queue_release = eth_queue_release,
.link_update = eth_link_update,
+ .queue_stats_mapping_set = eth_queue_stats_mapping_set,
.stats_get = eth_stats_get,
.stats_reset = eth_stats_reset,
.mac_addr_remove = eth_mac_addr_remove,
--
1.7.9.5
Tomasz Kulasek
2015-06-03 10:59:05 UTC
Permalink
This application allows you to test RSS configuration for bonded devices
changing configuration dynamically, during receiving packets on slaves and
observe the changes in its distribution over queues.

After initialization process, all accessible ports are attached to one bonding
as slaves.

Monitor screen is divided into five main parts:
- Port selection (on the very top)
- RSS Configuration for selected port including hash function, key and RETA
- Incoming packets statistics
- Incoming packets list for selected port
- Status bar with contextual information about selected part

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
examples/bond_rss/Makefile | 59 +++
examples/bond_rss/bondrss.c | 293 ++++++++++++++
examples/bond_rss/bondrss.h | 163 ++++++++
examples/bond_rss/config.c | 251 ++++++++++++
examples/bond_rss/ui.c | 915 +++++++++++++++++++++++++++++++++++++++++++
5 files changed, 1681 insertions(+)
create mode 100644 examples/bond_rss/Makefile
create mode 100644 examples/bond_rss/bondrss.c
create mode 100644 examples/bond_rss/bondrss.h
create mode 100644 examples/bond_rss/config.c
create mode 100644 examples/bond_rss/ui.c

diff --git a/examples/bond_rss/Makefile b/examples/bond_rss/Makefile
new file mode 100644
index 0000000..db457a3
--- /dev/null
+++ b/examples/bond_rss/Makefile
@@ -0,0 +1,59 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# binary name
+APP = bondrss
+
+# all source are stored in SRCS-y
+SRCS-y := bondrss.c
+SRCS-y += ui.c
+SRCS-y += config.c
+
+CFLAGS += $(WERROR_FLAGS)
+
+# workaround for a gcc bug with noreturn attribute
+# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603
+ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y)
+CFLAGS_main.o += -Wno-return-type
+endif
+
+LDLIBS += -lncurses
+
+include $(RTE_SDK)/mk/rte.extapp.mk
diff --git a/examples/bond_rss/bondrss.c b/examples/bond_rss/bondrss.c
new file mode 100644
index 0000000..2dc9979
--- /dev/null
+++ b/examples/bond_rss/bondrss.c
@@ -0,0 +1,293 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "bondrss.h"
+
+#define RSS_HASH_KEY_LENGTH 40
+
+static const struct rte_eth_conf port_conf_default = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_RSS,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled. */
+ .hw_ip_checksum = 0, /**< IP checksum offload disabled. */
+ .hw_vlan_filter = 1, /**< VLAN filtering enabled. */
+ .hw_vlan_strip = 1, /**< VLAN strip enabled. */
+ .hw_vlan_extend = 0, /**< Extended VLAN disabled. */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */
+ .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */
+ .enable_scatter = 0, /**< scatter rx disabled */
+ },
+ .rx_adv_conf = {
+ .rss_conf = {
+ .rss_key = NULL,
+ .rss_hf = ETH_RSS_IPV4,
+ },
+ },
+};
+
+static const struct rte_eth_conf port_conf_bonding = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_RSS,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled. */
+ .hw_ip_checksum = 0, /**< IP checksum offload disabled. */
+ .hw_vlan_filter = 1, /**< VLAN filtering enabled. */
+ .hw_vlan_strip = 1, /**< VLAN strip enabled. */
+ .hw_vlan_extend = 0, /**< Extended VLAN disabled. */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */
+ .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */
+ .enable_scatter = 0, /**< scatter rx disabled */
+ },
+ .rx_adv_conf = {
+ .rss_conf = {
+ .rss_key = NULL,
+ .rss_hf = ETH_RSS_IPV6,
+ },
+ },
+};
+
+/*
+ * Configurable number of RX/TX queues.
+ */
+int nb_rxq = 4; /**< Number of RX queues per port. */
+int nb_txq = 1; /**< Number of TX queues per port. */
+
+int bond_port_id = -1; /**< Bonded device port id */
+
+int nb_ports;
+struct rte_port *ports;
+rte_atomic64_t counter; /**< Received packet's counter */
+
+/*
+ * Initializes a given port using global settings and with the rx buffers
+ * coming from the mbuf_pool passed as parameter
+ */
+static inline
+int init_port(uint8_t port_id, struct rte_mempool *mbuf_pool,
+ struct rte_eth_conf port_conf)
+{
+
+ int retval;
+ uint16_t q;
+ struct rte_eth_rxconf rxconf;
+
+ struct rte_port *port;
+
+ if (port_id >= rte_eth_dev_count())
+ return -1;
+
+ port = &ports[port_id];
+ port->nb_rx_queues = nb_rxq;
+ port->nb_tx_queues = nb_txq;
+
+ memcpy(&(port->dev_conf), &port_conf, sizeof(port_conf));
+ retval = rte_eth_dev_configure(port_id, port->nb_rx_queues,
+ port->nb_tx_queues, &port_conf);
+
+ if (retval != 0)
+ return retval;
+
+ rte_eth_dev_info_get(port_id, &(port->dev_info));
+ rxconf = (port->dev_info).default_rxconf;
+ rxconf.rx_free_thresh = 32;
+
+ for (q = 0; q < port->nb_rx_queues; q++) {
+ retval = rte_eth_rx_queue_setup(port_id, q, RX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id), &rxconf,
+ mbuf_pool);
+ if (retval < 0)
+ return retval;
+ }
+
+ for (q = 0; q < port->nb_tx_queues; q++) {
+ retval = rte_eth_tx_queue_setup(port_id, q, TX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id),
+ &((port->dev_info).default_txconf));
+ if (retval < 0)
+ return retval;
+ }
+
+ port->bond_mode = -1;
+ port->promiscuous = 0;
+ port->enabled = 1;
+
+ return 0;
+}
+
+static int
+rx_loop(__attribute__((unused)) void *dummy)
+{
+ uint8_t port_id;
+ int nq;
+ int n;
+
+ for (;;)
+ for (port_id = 0; port_id < nb_ports; port_id++) {
+ /* Pool only bonding ports */
+ if (ports[port_id].bond_mode >= 0)
+ for (nq = 0; nq < ports[port_id].nb_rx_queues; nq++) {
+ struct rte_mbuf *bufs[BURST_SIZE];
+
+ const uint16_t nb_rx = rte_eth_rx_burst(port_id, nq, bufs,
+ BURST_SIZE);
+ if (unlikely(nb_rx == 0))
+ continue;
+
+ ports[port_id].rxq_ipackets[nq] += nb_rx;
+ for (n = 0; n < nb_rx; n++) {
+ record_pkt(port_id, nq, bufs[n]);
+ rte_pktmbuf_free(bufs[n]);
+ }
+ }
+ }
+
+ return 0;
+}
+
+static uint16_t
+rx_callback(uint8_t port_id, uint16_t qidx, struct rte_mbuf **pkts,
+ uint16_t nb_pkts, uint16_t max_pkts __rte_unused, void *_ __rte_unused)
+{
+ int n;
+
+ ports[port_id].rxq_ipackets[qidx] += nb_pkts;
+ for (n = 0; n < nb_pkts; n++) {
+ rte_atomic64_inc(&counter);
+ pkts[n]->udata64 = counter.cnt;
+ record_pkt(port_id, qidx, pkts[n]);
+ }
+
+ return nb_pkts;
+}
+
+int
+main(int argc, char *argv[])
+{
+ struct rte_mempool *mbuf_pool;
+ unsigned lcore_id = 0;
+ uint8_t port_id, queue_id;
+
+ /* init EAL */
+ int ret = rte_eal_init(argc, argv);
+ if (ret < 0)
+ rte_exit(EXIT_FAILURE, "Error with EAL initialization\n");
+ argc -= ret;
+ argv += ret;
+
+ nb_ports = rte_eth_dev_count();
+ if (nb_ports < 1)
+ rte_exit(EXIT_FAILURE,
+ "At least one port must be available to create a bonding\n");
+
+ mbuf_pool = rte_mempool_create("MBUF_POOL", NUM_MBUFS * nb_ports, MBUF_SIZE,
+ MBUF_CACHE_SIZE,
+ sizeof(struct rte_pktmbuf_pool_private),
+ rte_pktmbuf_pool_init, NULL,
+ rte_pktmbuf_init, NULL, rte_socket_id(), 0);
+ if (mbuf_pool == NULL)
+ rte_exit(EXIT_FAILURE, "Cannot create mbuf pool\n");
+
+ /* Configuration of Ethernet ports. */
+ ports = rte_zmalloc("bondrss: ports",
+ sizeof(struct rte_port) * RTE_MAX_ETHPORTS,
+ RTE_CACHE_LINE_SIZE);
+ if (ports == NULL)
+ rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) failed\n",
+ RTE_MAX_ETHPORTS);
+
+ /* enabled allocated ports */
+ for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++)
+ ports[port_id].enabled = 0;
+
+ /* initialize all ports */
+ for (port_id = 0; port_id < nb_ports; port_id++) {
+ if (init_port(port_id, mbuf_pool, port_conf_default) != 0)
+ rte_exit(EXIT_FAILURE, "Cannot init port %"PRIu8"\n", port_id);
+
+ /* Add rx callbacks to the slave's port */
+ for (queue_id = 0; queue_id < ports[port_id].nb_rx_queues; queue_id++)
+ rte_eth_add_rx_callback(port_id, queue_id, rx_callback, NULL);
+
+ }
+
+ /* create bonding port */
+ bond_port_id = rte_eth_bond_create("eth_bond", 0, 0);
+ if (bond_port_id < 0)
+ rte_exit(EXIT_FAILURE, "Cannot create bonding port\n");
+
+ for (port_id = 0; port_id < nb_ports; port_id++)
+ rte_eth_bond_slave_add(bond_port_id, port_id);
+
+ /* count again */
+ nb_ports = rte_eth_dev_count();
+
+ init_port(bond_port_id, mbuf_pool, port_conf_bonding);
+
+ /* start bonding port*/
+ ret = rte_eth_dev_start(bond_port_id);
+
+ /* enable promiscuous by default */
+ rte_eth_promiscuous_enable(bond_port_id);
+
+ for (port_id = 0; port_id < nb_ports; port_id++) {
+ ports[port_id].bond_mode = rte_eth_bond_mode_get(port_id);
+ ports[port_id].promiscuous = rte_eth_promiscuous_get(port_id);
+
+ /* map queues to stats */
+ if (ports[port_id].bond_mode >= 0) {
+ for (queue_id = 0; queue_id < ports[port_id].nb_rx_queues;
+ queue_id++) {
+ rte_eth_dev_set_rx_queue_stats_mapping(port_id, queue_id,
+ queue_id);
+ }
+ }
+
+ }
+
+ /* Initialize received packet's counter */
+ rte_atomic64_init(&counter);
+
+ if (rte_lcore_count() <= 1)
+ rte_exit(EXIT_FAILURE, "More than one free lcore is needed\n");
+
+ lcore_id = rte_get_next_lcore(rte_lcore_id(), 1, 0);
+ rte_eal_remote_launch(rx_loop, NULL, lcore_id);
+
+ /* call lcore_main on master core only */
+ ui_loop();
+
+ return 0;
+}
diff --git a/examples/bond_rss/bondrss.h b/examples/bond_rss/bondrss.h
new file mode 100644
index 0000000..3e2ef73
--- /dev/null
+++ b/examples/bond_rss/bondrss.h
@@ -0,0 +1,163 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef BONDRSS_H_
+#define BONDRSS_H_
+
+#include <stdint.h>
+#include <inttypes.h>
+#include <rte_eal.h>
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_lcore.h>
+#include <rte_mbuf.h>
+#include <rte_malloc.h>
+
+#include <rte_eth_bond.h>
+
+#define RX_RING_SIZE 128
+#define TX_RING_SIZE 512
+
+#define NUM_MBUFS 8191
+#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)
+#define MBUF_CACHE_SIZE 250
+#define BURST_SIZE 32
+
+/*
+ * Configurable number of RX/TX queues.
+ */
+extern int nb_rxq; /**< Number of RX queues per port. */
+extern int nb_txq; /**< Number of TX queues per port. */
+
+extern int bond_port_id; /**< Bonded device port id */
+extern int bond_mode;
+
+struct rss_type_info {
+ char str[32];
+ uint64_t rss_type;
+};
+
+static struct rss_type_info rss_type_table[] = {
+ {"ipv4", ETH_RSS_IPV4},
+ {"ipv4-frag", ETH_RSS_FRAG_IPV4},
+ {"ipv4-tcp", ETH_RSS_NONFRAG_IPV4_TCP},
+ {"ipv4-udp", ETH_RSS_NONFRAG_IPV4_UDP},
+ {"ipv4-sctp", ETH_RSS_NONFRAG_IPV4_SCTP},
+ {"ipv4-other", ETH_RSS_NONFRAG_IPV4_OTHER},
+ {"ipv6", ETH_RSS_IPV6},
+ {"ipv6-frag", ETH_RSS_FRAG_IPV6},
+ {"ipv6-tcp", ETH_RSS_NONFRAG_IPV6_TCP},
+ {"ipv6-udp", ETH_RSS_NONFRAG_IPV6_UDP},
+ {"ipv6-sctp", ETH_RSS_NONFRAG_IPV6_SCTP},
+ {"ipv6-other", ETH_RSS_NONFRAG_IPV6_OTHER},
+ {"l2-payload", ETH_RSS_L2_PAYLOAD},
+ {"ipv6-ex", ETH_RSS_IPV6_EX},
+ {"ipv6-tcp-ex", ETH_RSS_IPV6_TCP_EX},
+ {"ipv6-udp-ex", ETH_RSS_IPV6_UDP_EX},
+};
+
+struct rss_type_fn {
+ uint8_t enabled;
+ struct rss_type_info *info;
+};
+
+/**
+ * Buffer for last received packets
+ */
+#define PKT_RECORD_SIZE 256
+
+struct pkt_info {
+ uint64_t n; /**< Packet number */
+ uint8_t port_id;
+ uint8_t queue_id;
+ uint64_t ol_flags; /**< Offload features. */
+ uint32_t rss; /**< RSS hash result if RSS enabled */
+};
+
+struct pkt_record {
+ uint64_t count; /**< Total number of received packets */
+ int top;
+ struct pkt_info pkt_info[PKT_RECORD_SIZE];
+};
+
+/**
+ * The data structure associated with each port.
+ */
+struct rte_port {
+ struct rte_eth_dev_info dev_info; /**< PCI info + driver name */
+ struct rte_eth_conf dev_conf; /**< Port configuration. */
+
+ uint16_t nb_rx_queues; /**< Total number of rx queues */
+ uint16_t nb_tx_queues; /**< Total number of tx queues*/
+
+ int bond_mode;
+ struct ether_addr addr;
+
+ int rss_type_count;
+ struct rss_type_fn rss_type_fn[RTE_DIM(rss_type_table)];
+ struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE];
+ uint8_t rss_key[40];
+
+
+ struct rte_eth_stats stats;
+ uint64_t rxq_ipackets[128];
+
+ struct pkt_record record;
+
+ uint8_t rss; /**< RSS enabled for port */
+ uint8_t promiscuous; /**< promiscuous mode enabled for port */
+ uint8_t enabled; /**< port is enabled */
+};
+
+extern int nb_ports;
+extern struct rte_port *ports;
+extern rte_atomic64_t counter; /**< Received packet's counter */
+
+void update_port_info(uint8_t port_id);
+
+int rss_enable(uint8_t port_id, int enabled);
+
+int key_set(uint8_t port_id, uint8_t *key, uint8_t len);
+int key_set_random(uint8_t port_id, uint8_t len);
+
+int reta_set_default(uint8_t port_id);
+int reta_set_random(uint8_t port_id);
+int reta_set_all(uint8_t port_id, uint8_t value);
+int reta_set_weights(uint8_t port_id, uint64_t *weights);
+
+struct pkt_info *record_pkt(uint8_t port_id, int queue_id, struct rte_mbuf *mb);
+
+void ui_loop(void);
+
+
+#endif /* BONDRSS_H_ */
diff --git a/examples/bond_rss/config.c b/examples/bond_rss/config.c
new file mode 100644
index 0000000..39c00e5
--- /dev/null
+++ b/examples/bond_rss/config.c
@@ -0,0 +1,251 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <time.h>
+#include <stdlib.h>
+#include "bondrss.h"
+
+static const struct rte_eth_conf port_conf_default = {
+ .rxmode = {
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled. */
+ .hw_ip_checksum = 0, /**< IP checksum offload disabled. */
+ .hw_vlan_filter = 1, /**< VLAN filtering enabled. */
+ .hw_vlan_strip = 1, /**< VLAN strip enabled. */
+ .hw_vlan_extend = 0, /**< Extended VLAN disabled. */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */
+ .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */
+ .enable_scatter = 0, /**< scatter rx disabled */
+ },
+ .rx_adv_conf = {
+ .rss_conf = {
+ .rss_key = NULL,
+ .rss_hf = ETH_RSS_IPV4,
+ },
+ },
+};
+
+
+void
+update_port_info(uint8_t port_id)
+{
+ int i;
+ struct rte_port *port;
+
+ port = &ports[port_id];
+
+ rte_eth_dev_info_get(port_id, &(port->dev_info));
+
+ port->promiscuous = rte_eth_promiscuous_get(port_id);
+
+ /* Update device information */
+ rte_eth_dev_info_get(port_id, &port->dev_info);
+ rte_eth_macaddr_get(port_id, &port->addr);
+
+ port->dev_conf.rx_adv_conf.rss_conf.rss_key = port->rss_key;
+ port->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 40;
+ rte_eth_dev_rss_hash_conf_get(port_id,
+ &(port->dev_conf.rx_adv_conf.rss_conf));
+
+ /* select all fields to be fetched */
+ for (i = 0; i < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; i++)
+ port->reta_conf[i].mask = ~0LL;
+
+ rte_eth_dev_rss_reta_query(port_id, port->reta_conf,
+ port->dev_info.reta_size);
+
+}
+
+/**
+ * Try to enable RSS for port_id.
+ */
+int
+rss_enable(uint8_t port_id, int enabled)
+{
+ struct rte_port *port;
+ int retval;
+
+ port = &ports[port_id];
+
+ rte_eth_dev_stop(port_id);
+
+ if (enabled) {
+ port->dev_conf.rxmode.mq_mode = ETH_MQ_RX_RSS;
+ port->dev_conf.rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IPV4;
+ } else {
+ port->dev_conf.rxmode.mq_mode = 0;
+ port->dev_conf.rx_adv_conf.rss_conf.rss_hf = 0;
+ }
+
+ retval = rte_eth_dev_configure(port_id, port->nb_rx_queues,
+ port->nb_tx_queues, &port->dev_conf);
+ if (retval != 0)
+ return retval;
+
+ retval = rte_eth_dev_start(port_id);
+
+ return 0;
+}
+
+int
+key_set(uint8_t port_id, uint8_t *key, uint8_t len)
+{
+ struct rte_eth_rss_conf rss_conf;
+
+ int i;
+
+ rss_conf.rss_key = NULL;
+ rss_conf.rss_key_len = 0;
+ i = rte_eth_dev_rss_hash_conf_get(port_id, &rss_conf);
+
+ rss_conf.rss_key = key;
+ rss_conf.rss_key_len = len;
+
+ i = rte_eth_dev_rss_hash_update(port_id,
+ &rss_conf);
+
+ return i;
+}
+
+/**
+ * Set random RSS key value
+ */
+int
+key_set_random(uint8_t port_id, uint8_t len)
+{
+ int i;
+ uint8_t key[40];
+
+ for (i = 0; i < len; i++)
+ key[i] = rand() % 256;
+
+ return key_set(port_id, key, len);
+}
+
+/**
+ * Set random RETA values
+ */
+int
+reta_set_random(uint8_t port_id)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ int i, j;
+
+ int reta_size = ports[port_id].dev_info.reta_size;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) {
+ /* select all fields to set */
+ reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ reta_conf[i].reta[j] = rand() % ports[port_id].nb_rx_queues;
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
+
+/**
+ * Set default RETA values
+ */
+int
+reta_set_default(uint8_t port_id)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ int i, j;
+
+ int reta_size = ports[port_id].dev_info.reta_size;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) {
+ /* select all fields to set */
+ reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ reta_conf[i].reta[j] = j % nb_rxq;
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
+
+/**
+ * Fill the RETA with values
+ */
+int
+reta_set_all(uint8_t port_id, uint8_t value)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ int i, j;
+
+ int reta_size = ports[port_id].dev_info.reta_size;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) {
+ /* select all fields to set */
+ reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ reta_conf[i].reta[j] = value;
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
+
+/**
+ * Set the RETA basing on weights table
+ */
+int
+reta_set_weights(uint8_t port_id, uint64_t *weights)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ unsigned i, j, k;
+
+ unsigned reta_size = ports[port_id].dev_info.reta_size;
+
+ uint64_t sum = 0, sum2;
+
+ for (i = 0; i < ports[port_id].nb_rx_queues; i++)
+ sum += weights[i];
+ sum2 = sum * (ports[port_id].nb_rx_queues-1);
+
+ if (sum2 == 0)
+ return 0;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++)
+ reta_conf[i].mask = ~0LL;
+ k = 0;
+ for (i = 0; i < ports[port_id].nb_rx_queues; i++) {
+ for (j = 0; k < reta_size &&
+ j < ((sum - weights[i]) * reta_size + 1) / sum2; j++) {
+ reta_conf[k/RTE_RETA_GROUP_SIZE].reta[k%RTE_RETA_GROUP_SIZE] = i;
+ k++;
+ }
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
diff --git a/examples/bond_rss/ui.c b/examples/bond_rss/ui.c
new file mode 100644
index 0000000..be3767c
--- /dev/null
+++ b/examples/bond_rss/ui.c
@@ -0,0 +1,915 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_atomic.h>
+
+#include <ncurses.h>
+#include <string.h>
+#include <signal.h>
+#include <sys/ioctl.h>
+#include <menu.h>
+#include "bondrss.h"
+
+WINDOW *win_footer = NULL;
+WINDOW *win_header = NULL;
+WINDOW *win_list = NULL;
+WINDOW *win_stats = NULL;
+WINDOW *win_reta = NULL;
+WINDOW *win_err = NULL;
+
+int scr_lines = 0;
+int scr_cols = 0;
+
+unsigned win_header_lines;
+unsigned win_list_lines;
+unsigned win_stats_lines;
+unsigned win_reta_lines;
+unsigned win_err_lines = 0;
+
+const char *ui_status_text;
+static char *ui_err_text[5];
+
+struct rte_port *selected_port;
+static uint8_t selected_port_id;
+
+struct rte_port *focused_port;
+static uint8_t focused_port_id;
+
+#define UI_MODE_PORT 0
+#define UI_MODE_RSS_FN 1
+#define UI_MODE_RSS_KEY 2
+#define UI_MODE_RETA 3
+#define UI_MODE_HELP 4
+
+int ui_mode = UI_MODE_PORT;
+
+static uint16_t _is_lock;
+static uint16_t _is_ready;
+
+static uint16_t _do_configure;
+static uint16_t _do_repaint;
+static uint16_t _do_repaint_stats;
+static uint16_t _do_repaint_list;
+static uint16_t _do_repaint_info;
+static uint16_t _do_repaint_err;
+
+
+struct help_page {
+ const char *title;
+ const char *body;
+};
+
+static struct help_page help_pages[3] = {
+ {
+ .title = "About",
+
+ .body =
+ "\n\n"
+ " ______ ______ ______ _____ ______ ______ ______\n"
+ "| | | \\ / | | \\ | | \\ \\ | | \\ \\ | | | \\ / | / |\n"
+ "| |--| < | | | | | | | | | | | | | |__| | '------. '------.\n"
+ "|_|__|_/ \\_|__|_/ |_| |_| |_|_/_/ |_| \\_\\ ____|_/ ____|_/\n"
+ "\n\n"
+ "This application allows you to test RSS configuration for "
+ "bonded devices\nchanging configuration dynamically, during "
+ "receiving packets on slaves and\nobserve the changes in its "
+ "distribution over queues.\n"
+ "\n"
+
+ },
+ {
+ .title = "Introduction",
+ .body = "\n\n"
+ "After initialization process, all accessible ports are "
+ "attached to one\nbonding as slaves.\n"
+ "\n"
+
+ "Monitor screen is divided into five main parts:\n"
+ " - Port selection (on the very top)\n"
+ " - RSS Configuration for selected port including hash "
+ "function, key and\n RETA\n"
+ " - Incoming packets statistics\n"
+ " - Incoming packets list for selected port\n"
+ " - Status bar with contextual information about selected "
+ "part\n"
+ "\n\n"
+
+ "[up], [down] arrows key lets you highlight a part "
+ "and parameter to change.\n\n"
+ },
+ {
+ .title = "Key bindings",
+ .body =
+ "_Port_selection_\n\n"
+ " [left], [right] - select port,\n"
+ " [space] - force turning on/off RSS mode,\n"
+ " [p] - turn on/off promiscuous mode,\n"
+ " [c] - clear statistics and incoming packet list,\n"
+ "\n"
+ "_RSS_hash_function_\n\n"
+ " [1]-[9] - toggle selected hash function,\n"
+ " [a] - select all,\n"
+ "\n"
+ "_RSS_key_\n\n"
+ " [1]-[4] - set one of predefined key value,\n"
+ " [r] - randomize value,\n"
+ "\n"
+ "_RSS_RETA_\n\n"
+ " [number] - fill RETA with number,\n"
+ " [r] - randomize value,\n"
+ " [d] - set default value,"
+
+ }
+};
+
+static int
+ui_lock(int wait)
+{
+ int result;
+
+ while (!(result = rte_atomic16_cmpset(&_is_lock, 0, 1)) && wait)
+ rte_pause();
+
+ return result;
+}
+
+static int
+ui_unlock(void)
+{
+ return rte_atomic16_cmpset(&_is_lock, 1, 0);
+}
+
+/**
+ * Predefined key values
+ */
+static uint8_t RSS_KEY[4][40] = {
+ { 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D,
+ 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+ 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B,
+ 0xBE, 0xAC, 0x01, 0xFA
+ },
+ { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00,
+ },
+ { 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A,
+ 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A,
+ 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A,
+ 0x6D, 0x5A, 0x6D, 0x5A,
+ },
+ { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C,
+ 0x0D, 0x0E, 0x0F, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18,
+ 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F, 0x20, 0x21, 0x22, 0x23, 0x24,
+ 0x25, 0x26, 0x27, 0x28,
+ }
+};
+
+static void
+win_err_addline(const char *text)
+{
+ int i;
+
+ if (win_err_lines < 5) {
+ win_err_lines++;
+ _do_configure = 1;
+ } else {
+ free(ui_err_text[0]);
+ for (i = 1; i < 5; i++)
+ ui_err_text[i - 1] = ui_err_text[i];
+ }
+
+ ui_err_text[win_err_lines - 1] = strdup(text);
+ _do_repaint_err = 1;
+}
+
+static void
+win_err_fini(void)
+{
+ int i;
+ for (i = 0; i < (int)win_err_lines; i++)
+ free(ui_err_text[i]);
+}
+
+/**
+ * Draw full information for port
+ */
+static void
+win_header_draw(uint8_t port_id)
+{
+ int i, j;
+ attr_t a;
+
+ update_port_info(selected_port_id);
+
+ /* Draw title bar */
+ if (ui_mode == UI_MODE_PORT)
+ wattron(win_header, COLOR_PAIR(2));
+ else
+ wattron(win_header, COLOR_PAIR(4));
+
+ mvwprintw(win_header, 0, 0, "::");
+
+ for (i = 0; i < nb_ports; i++) {
+ if (ports[i].enabled) {
+ const char *name = (ports[i].bond_mode >= 0 ? "BOND" : "SLAVE");
+
+ if (i == port_id) {
+ wattron(win_header, A_REVERSE);
+ wprintw(win_header, " %s-%d ", name, (unsigned) i);
+ wattroff(win_header, A_REVERSE);
+ } else
+ wprintw(win_header, " %s-%d ", name, (unsigned) i);
+ }
+ }
+
+ for (i = scr_cols - getcurx(win_header) - 3; i > 0; i--)
+ waddch(win_header, ':');
+
+ waddch(win_header,
+ (ports[port_id].rss ? 'R' : '-'));
+ waddch(win_header, ports[port_id].bond_mode >= 0 ?
+ ports[port_id].bond_mode + '0' : '-');
+ waddch(win_header, ports[port_id].promiscuous == 0 ? '-' : 'P');
+
+ if (ui_mode == UI_MODE_PORT)
+ wattroff(win_header, COLOR_PAIR(2));
+ else
+ wattroff(win_header, COLOR_PAIR(4));
+
+ /* Redraw RSS-Fn */
+ selected_port->rss_type_count = 0;
+ for (i = 0; i < (int) RTE_DIM(rss_type_table); i++) {
+ if (selected_port->dev_info.flow_type_rss_offloads
+ & rss_type_table[i].rss_type) {
+ selected_port->rss_type_fn[selected_port->rss_type_count].info =
+ &rss_type_table[i];
+ selected_port->rss_type_fn[selected_port->rss_type_count].enabled =
+ ((selected_port->dev_conf.rx_adv_conf.rss_conf.rss_hf
+ & rss_type_table[i].rss_type) ? 1 : 0);
+ selected_port->rss_type_count++;
+ }
+ }
+
+ a = (ui_mode == UI_MODE_RSS_FN ? COLOR_PAIR(2) : COLOR_PAIR(1));
+
+ wattron(win_header, a);
+ mvwprintw(win_header, 1, 0, "FN: ");
+ for (i = 0; i < selected_port->rss_type_count; i++) {
+ if (selected_port->rss_type_fn[i].enabled)
+ wattron(win_header, COLOR_PAIR(3));
+ waddstr(win_header, selected_port->rss_type_fn[i].info->str);
+ if (selected_port->rss_type_fn[i].enabled)
+ wattron(win_header, a);
+ waddch(win_header, ' ');
+ }
+ wattroff(win_header, a);
+
+ /* Redraw RSS-Key */
+ if (ui_mode == UI_MODE_RSS_KEY)
+ wattron(win_header, COLOR_PAIR(2));
+ mvwprintw(win_header, 2, 0, "KEY: ");
+ for (i = 0; i < 40; i++)
+ wprintw(win_header, "%02X",
+ selected_port->dev_conf.rx_adv_conf.rss_conf.rss_key[i]);
+ if (ui_mode == UI_MODE_RSS_KEY)
+ wattroff(win_header, COLOR_PAIR(2));
+
+
+ /* Redraw RETA window */
+ int idx, shift;
+
+ if (ui_mode == UI_MODE_RETA)
+ wattron(win_reta, COLOR_PAIR(2));
+
+ for (j = 0; j < selected_port->dev_info.reta_size / 16; j++) {
+ wmove(win_reta, j, 0);
+ for (i = 0; i < 16; i++) {
+ idx = (j*16 + i) / RTE_RETA_GROUP_SIZE;
+ shift = (j*16 + i) % RTE_RETA_GROUP_SIZE;
+ waddch(win_reta, ACS_VLINE);
+ wprintw(win_reta, "%d", selected_port->reta_conf[idx].reta[shift]);
+ }
+ waddch(win_reta, ACS_VLINE);
+ }
+
+ if (ui_mode == UI_MODE_RETA)
+ wattroff(win_reta, COLOR_PAIR(2));
+ wnoutrefresh(win_reta);
+
+
+ /* Stats repaint */
+ if (_do_repaint_stats) {
+ uint64_t total;
+
+ rte_eth_stats_get(selected_port_id, &(selected_port->stats));
+
+ wmove(win_stats, 0, 0);
+ total = 0;
+ for (i = 0; i < selected_port->nb_rx_queues; i++) {
+ wprintw(win_stats, "Queue %d: %10"PRIu64
+ " (%10" PRIu64 ")\n", i, selected_port->rxq_ipackets[i],
+ selected_port->stats.q_ipackets[i]);
+ total += selected_port->rxq_ipackets[i];
+ }
+
+ wprintw(win_stats, " Total: %10"PRIu64
+ " (%10" PRIu64 ")\n", total, selected_port->stats.ipackets);
+
+ _do_repaint_stats = 0;
+ wnoutrefresh(win_stats);
+ }
+
+ if (_do_repaint_err && win_err_lines > 0) {
+ for (i = 0; i < (int)win_err_lines; i++) {
+ mvwprintw(win_err, i, 0, ui_err_text[i]);
+ wclrtoeol(win_err);
+ }
+ _do_repaint_err = 0;
+ wnoutrefresh(win_err);
+ }
+
+ mvwhline(win_header, win_header_lines - 1, 0, 0, scr_cols);
+}
+
+static void
+win_footer_draw(const char *text)
+{
+ if (text != NULL)
+ ui_status_text = text;
+
+ wclear(win_footer);
+ wmove(win_footer, 0, 0);
+ wprintw(win_footer, ui_status_text);
+
+ wprintw(win_footer, "; [arrows]navigate; [q]quit");
+ wnoutrefresh(win_footer);
+}
+
+static void
+win_list_draw(void)
+{
+ unsigned n, i;
+
+ struct pkt_info *pkt_info;
+
+ struct rte_port *port = &ports[focused_port_id];
+ struct pkt_record *record = &(port->record);
+
+ n = (win_list_lines > record->count) ? record->count : win_list_lines;
+
+ wmove(win_list, 0, 0);
+
+ for (i = 0; i < n; i++) {
+
+ pkt_info = &record->pkt_info[(record->top - n + i + PKT_RECORD_SIZE)
+ % PKT_RECORD_SIZE];
+
+ wmove(win_list, i, 0);
+
+ wprintw(win_list, "%5"PRIu64, (unsigned) pkt_info->n);
+ waddch(win_list, ACS_VLINE);
+
+ wprintw(win_list, "%2d: ", (unsigned) pkt_info->queue_id);
+ wprintw(win_list, "%08x ", (unsigned) pkt_info->rss);
+
+ if (pkt_info->ol_flags != 0) {
+ unsigned rxf;
+ const char *name;
+
+ for (rxf = 0; rxf < sizeof(pkt_info->ol_flags) * 8; rxf++) {
+ if ((pkt_info->ol_flags & (1ULL << rxf)) == 0)
+ continue;
+ name = rte_get_rx_ol_flag_name(1ULL << rxf);
+ if (name == NULL)
+ continue;
+ wprintw(win_list, "%s ", name);
+ }
+ }
+ wclrtoeol(win_list);
+ }
+
+ wclrtobot(win_list);
+ wnoutrefresh(win_list);
+}
+
+static void
+ui_repaint(void)
+{
+ switch (ui_mode) {
+ default:
+ win_header_draw(selected_port_id);
+ wnoutrefresh(win_header);
+
+ if (_do_repaint_list) {
+ win_list_draw();
+ _do_repaint_list = 0;
+ }
+
+ break;
+ }
+
+ win_footer_draw(NULL);
+ _do_repaint = 0;
+}
+
+static void
+ui_mode_set(int mode)
+{
+ ui_mode = (mode + 4) % 4;
+ switch (ui_mode) {
+ case UI_MODE_PORT:
+ ui_status_text = "Port: [p]promiscuous; [c]clear stats";
+ break;
+ case UI_MODE_RSS_FN:
+ ui_status_text = "RSS Fn: [1-9]toggle fn";
+ break;
+ case UI_MODE_RETA:
+ ui_status_text =
+ "RETA: [d]set default; [r]randomize; [0-9]fill table with value";
+ break;
+ case UI_MODE_RSS_KEY:
+ ui_status_text = "RSS Key: [r]randomize; [1-4]select predefined key";
+ break;
+ }
+ _do_repaint = 1;
+}
+
+static void
+ui_configure(void)
+{
+ int win_reta_cols = (16 * 2 + 1);
+ win_reta_lines = selected_port->dev_info.reta_size / 16;
+ win_stats_lines = selected_port->nb_rx_queues + 1;
+ win_header_lines = win_reta_lines;
+ if (win_header_lines < win_stats_lines)
+ win_header_lines = win_stats_lines;
+ win_header_lines += 5;
+ win_list_lines = scr_lines - win_header_lines - win_err_lines - 1;
+
+ if (win_footer == NULL) {
+ win_footer = newwin(1, scr_cols, scr_lines - 1, 0);
+ ui_status_text = "";
+ wbkgdset(win_footer, COLOR_PAIR(1));
+ } else {
+ wresize(win_footer, 1, scr_cols);
+ mvwin(win_footer, scr_lines - 1, 0);
+ }
+
+ if (win_err == NULL) {
+ win_err = newwin(1, scr_cols, scr_lines - 1 - win_err_lines, 0);
+ ui_status_text = "";
+ wbkgdset(win_err, COLOR_PAIR(5));
+ } else {
+ wresize(win_err, win_err_lines, scr_cols);
+ mvwin(win_err, scr_lines - win_err_lines - 1, 0);
+ }
+
+
+ if (win_header == NULL) {
+ win_header = newwin(win_header_lines, scr_cols, 0, 0);
+ wbkgdset(win_header, COLOR_PAIR(1));
+ } else
+ wresize(win_header, win_header_lines, scr_cols);
+
+ if (win_stats == NULL)
+ win_stats = subwin(win_header, win_stats_lines,
+ scr_cols - win_reta_cols - 7, 4, win_reta_cols + 7);
+ else
+ wresize(win_stats, win_stats_lines, scr_cols - win_reta_cols - 7);
+
+ if (win_reta == NULL)
+ win_reta = subwin(win_header, win_reta_lines, win_reta_cols,
+ 4, 5);
+ else
+ wresize(win_reta, win_reta_lines, win_reta_cols);
+
+ if (win_list == NULL)
+ win_list = newwin(win_list_lines, scr_cols, win_header_lines, 0);
+ else {
+ wresize(win_list, win_list_lines, scr_cols);
+ mvwin(win_list, win_header_lines, 0);
+ }
+
+ wclear(win_header);
+ wclear(win_reta);
+
+ _do_configure = 0;
+ _do_repaint = 1;
+ _do_repaint_list = 1;
+ _do_repaint_stats = 1;
+ _do_repaint_info = 1;
+ _do_repaint_err = 1;
+
+}
+
+static void
+ui_resize(void)
+{
+ if (COLS != scr_cols || LINES != scr_lines) {
+ scr_cols = COLS;
+ scr_lines = LINES;
+ _do_configure = 1;
+ }
+}
+
+static void
+ui_update(void)
+{
+ if (ui_lock(1)) {
+ if (_is_ready) {
+ if (_do_configure)
+ ui_configure();
+ if (_do_repaint) {
+ ui_repaint();
+ doupdate();
+ }
+ }
+ ui_unlock();
+ }
+}
+
+static void
+ui_fini(void)
+{
+ win_err_fini();
+ endwin();
+}
+
+static void
+ui_init(void)
+{
+ initscr();
+ start_color();
+ init_pair(1, COLOR_YELLOW, COLOR_BLUE);
+ init_pair(2, COLOR_BLACK, COLOR_CYAN);
+ init_pair(3, COLOR_BLACK, COLOR_YELLOW);
+ init_pair(4, COLOR_BLACK, COLOR_WHITE);
+ init_pair(5, COLOR_YELLOW, COLOR_RED);
+
+ cbreak();
+ noecho();
+ curs_set(FALSE);
+ keypad(stdscr, TRUE);
+ timeout(100);
+
+ scr_cols = COLS;
+ scr_lines = LINES;
+ _do_configure = 1;
+ _is_ready = 1;
+}
+
+static int
+port_select(uint8_t port_id)
+{
+ if (ports[port_id].enabled == 1) {
+ focused_port_id = selected_port_id = port_id;
+
+ selected_port = &ports[selected_port_id];
+ focused_port = &ports[focused_port_id];
+
+ update_port_info(selected_port_id);
+
+ _do_configure = 1;
+ _do_repaint = 1;
+ _do_repaint_info = 1;
+ _do_repaint_list = 1;
+ _do_repaint_stats = 1;
+ return selected_port_id;
+ }
+ return -1;
+}
+
+/**
+ * Record packet from mbuf
+ */
+struct pkt_info *
+record_pkt(uint8_t port_id, int queue_id, struct rte_mbuf *mb)
+{
+ struct rte_port *port = &ports[port_id];
+ struct pkt_record *record = &(port->record);
+ struct pkt_info *pkt_info = &(record->pkt_info[record->top]);
+
+ ui_lock(1);
+
+ record->count++;
+ pkt_info->n = mb->udata64;
+ pkt_info->ol_flags = mb->ol_flags;
+ pkt_info->queue_id = queue_id;
+ pkt_info->rss = mb->hash.rss;
+
+ record->top = (record->top + 1) % PKT_RECORD_SIZE;
+
+ _do_repaint = 1;
+ _do_repaint_list = 1;
+ _do_repaint_stats = 1;
+ ui_unlock();
+
+ return pkt_info;
+}
+
+static void
+ui_help(void) {
+ timeout(-1);
+
+ WINDOW *win_help_border = NULL;
+ WINDOW *win_help = NULL;
+ char title[256];
+ int page = 0;
+ int page_prev;
+ int page_next;
+ int c;
+ int cols;
+ int lines;
+ int top;
+ int left;
+
+ endwin();
+ initscr();
+ ui_resize();
+ ui_update();
+
+ cols = scr_cols > 80 ? 80 : scr_cols;
+ lines = scr_lines > 25 ? 25 : scr_lines;
+ top = (scr_lines - lines) / 2;
+ left = (scr_cols - cols) / 2;
+
+ win_help_border = newwin(lines, cols, top, left);
+ win_help = derwin(win_help_border, lines - 2, cols - 4, 1, 2);
+ wbkgdset(win_help_border, COLOR_PAIR(2));
+ wbkgdset(win_help, COLOR_PAIR(2));
+
+ do {
+ page_prev = (page + RTE_DIM(help_pages) - 1) % RTE_DIM(help_pages);
+ page_next = (page + 1) % RTE_DIM(help_pages);
+
+ wclear(win_help_border);
+ box(win_help_border, 0 , 0);
+
+ sprintf(title, "[ Help: %s ]", help_pages[page].title);
+ mvwprintw(win_help_border, 0, (cols - strlen(title)) / 2 - 2, "%s",
+ title);
+
+ sprintf(title, "< %s ]", help_pages[page_prev].title);
+ mvwprintw(win_help_border, lines - 1, 1, "%s", title);
+
+ sprintf(title, "[ Page %d of %d ]", page + 1, (int)RTE_DIM(help_pages));
+ mvwprintw(win_help_border, lines - 1, (cols - strlen(title)) / 2 - 2,
+ "%s", title);
+
+ sprintf(title, "[ %s >", help_pages[page_next].title);
+ mvwprintw(win_help_border, lines - 1, cols - strlen(title) - 1, "%s",
+ title);
+
+ wrefresh(win_help_border);
+
+ wclear(win_help);
+
+ wmove(win_help, 0, 0);
+
+ wprintw(win_help,
+ help_pages[page].body
+ );
+
+ wrefresh(win_help);
+
+ switch (c = getch()) {
+
+ case KEY_RESIZE:
+ endwin();
+ initscr();
+ ui_resize();
+ ui_update();
+
+ cols = scr_cols > 80 ? 80 : scr_cols;
+ lines = scr_lines > 25 ? 25 : scr_cols;
+ top = (scr_lines - lines) / 2;
+ left = (scr_cols - cols) / 2;
+
+ wresize(win_help_border, lines, cols);
+ wresize(win_help, lines - 2, cols - 4);
+ break;
+
+ case KEY_LEFT:
+ page = page_prev;
+ break;
+
+ case KEY_RIGHT:
+ page = page_next;
+ break;
+
+ default:
+ c = -1; /* Exit */
+ break;
+ }
+
+
+ } while (c != -1);
+
+ timeout(100);
+ delwin(win_help);
+ delwin(win_help_border);
+ _do_configure = 1;
+ _do_repaint = 1;
+}
+
+/* main processing loop */
+void
+ui_loop(void)
+{
+ int c;
+ int i;
+ int port_id;
+
+ ui_init();
+ port_select(bond_port_id);
+
+ ui_mode_set(UI_MODE_PORT);
+
+ while ((c = getch()) != 'q') {
+ switch (c) {
+ case -1:
+ ui_update();
+ break;
+
+ case 'c':
+ /* clear stats */
+ ui_lock(1);
+
+ rte_atomic64_clear(&counter);
+ for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) {
+ if (ports[port_id].enabled) {
+ ports[port_id].record.count = 0;
+ ports[port_id].record.top = 0;
+ for (i = 0; i < ports[port_id].nb_rx_queues; i++)
+ ports[port_id].rxq_ipackets[i] = 0;
+ rte_eth_stats_reset(port_id);
+ }
+ }
+
+ _do_repaint = 1;
+ ui_unlock();
+ break;
+
+ case ' ':
+ focused_port->rss ^= 1;
+ rss_enable(focused_port_id, focused_port->rss);
+ _do_configure = 1;
+ _do_repaint = 1;
+ break;
+
+ case 'p':
+ if (focused_port->promiscuous)
+ rte_eth_promiscuous_disable(focused_port_id);
+ else
+ rte_eth_promiscuous_enable(focused_port_id);
+ focused_port->promiscuous ^= 1;
+ _do_configure = 1;
+ _do_repaint = 1;
+ break;
+
+ case KEY_RESIZE:
+ ui_resize();
+ break;
+
+ case KEY_UP:
+ ui_mode_set(ui_mode - 1);
+ break;
+ case KEY_DOWN:
+ ui_mode_set(ui_mode + 1);
+ break;
+ case 'h':
+ case '?':
+ ui_help();
+ break;
+
+ default:
+ switch (ui_mode) {
+ case UI_MODE_PORT:
+ switch (c) {
+ case KEY_LEFT:
+ for (i = 1; i < nb_ports; i++)
+ if (port_select(
+ (selected_port_id - i + nb_ports) % nb_ports)
+ != -1)
+ break;
+ break;
+
+ case KEY_RIGHT:
+ for (i = 1; i < nb_ports; i++)
+ if (port_select(
+ (selected_port_id + i + nb_ports) % nb_ports)
+ != -1)
+ break;
+ break;
+
+ default:
+ i = (c - '1');
+ port_select(i);
+ break;
+ }
+ break;
+
+ case UI_MODE_RSS_KEY:
+ switch (c) {
+ case 'r':
+ i = key_set_random(focused_port_id, 40);
+ if (i < 0)
+ win_err_addline("Cannot update RSS key");
+ break;
+
+ default:
+ c = c - '1';
+ if (c >= 0 && c < (int) RTE_DIM(RSS_KEY)) {
+ i = key_set(focused_port_id, RSS_KEY[c], 40);
+ if (i < 0)
+ win_err_addline("Cannot update RSS key");
+ }
+ break;
+ }
+ update_port_info(focused_port_id);
+ break;
+
+ case UI_MODE_RETA:
+ switch (c) {
+ case 'r':
+ reta_set_random(focused_port_id);
+ break;
+ case 'd':
+ reta_set_default(focused_port_id);
+ break;
+ default:
+ c = c - '0';
+ reta_set_all(focused_port_id, c);
+ break;
+ }
+ break;
+
+ case UI_MODE_RSS_FN:
+ switch (c) {
+ case 'a':
+ /* Set all */
+ focused_port->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ ports[focused_port_id].dev_info.flow_type_rss_offloads;
+ rte_eth_dev_rss_hash_update(focused_port_id,
+ &focused_port->dev_conf.rx_adv_conf.rss_conf);
+ break;
+
+ default:
+ c -= '1';
+ if (c >= 0 && c < focused_port->rss_type_count) {
+ uint64_t rss_type = 0;
+
+ focused_port->rss_type_fn[c].enabled ^= 1; /* toggle */
+
+ for (i = 0; i < focused_port->rss_type_count; i++)
+ if (focused_port->rss_type_fn[i].enabled)
+ rss_type |=
+ focused_port->rss_type_fn[i].info->rss_type;
+ focused_port->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ rss_type;
+
+ rte_eth_dev_rss_hash_update(focused_port_id,
+ &focused_port->dev_conf.rx_adv_conf.rss_conf);
+
+ }
+ }
+ break;
+
+ }
+ _do_repaint = 1;
+ break;
+ }
+ }
+ ui_fini();
+}
--
1.7.9.5
Tomasz Kulasek
2015-06-03 10:59:06 UTC
Permalink
Signed-off-by: Tomasz Kulasek <***@intel.com>
---
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
index 96e554f..03baf90 100644
--- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
+++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
@@ -188,7 +188,7 @@ conditions are not met. If a user wishes to monitor individual slaves then they
must register callbacks with that slave directly.

The link bonding library also supports devices which do not implement link
-status change interrupts, this is achieve by polling the devices link status at
+status change interrupts, this is achieved by polling the devices link status at
a defined period which is set using the ``rte_eth_bond_link_monitoring_set``
API, the default polling interval is 10ms. When a device is added as a slave to
a bonding device it is determined using the ``RTE_PCI_DRV_INTR_LSC`` flag
@@ -286,7 +286,7 @@ and UDP protocols for load balancing.
Using Link Bonding Devices
--------------------------

-The librte_pmd_bond library support two modes of device creation, the libraries
+The librte_pmd_bond library supports two modes of device creation, the libraries
export full C API or using the EAL command line to statically configure link
bonding devices at application startup. Using the EAL option it is possible to
use link bonding functionality transparently without specific knowledge of the
@@ -299,7 +299,7 @@ Using the Poll Mode Driver from an Application

Using the librte_pmd_bond libraries API it is possible to dynamically create
and manage link bonding device from within any application. Link bonding
-device are created using the ``rte_eth_bond_create`` API which requires a
+devices are created using the ``rte_eth_bond_create`` API which requires a
unique device name, the link bonding mode to initial the device in and finally
the socket Id which to allocate the devices resources onto. After successful
creation of a bonding device it must be configured using the generic Ethernet
@@ -362,7 +362,7 @@ The different options are:
mode=2

* slave: Defines the PMD device which will be added as slave to the bonded
- device. This option can be selected multiple time, for each device to be
+ device. This option can be selected multiple times, for each device to be
added as a slave. Physical devices should be specified using their PCI
address, in the format domain:bus:devid.function
--
1.7.9.5
Tomasz Kulasek
2015-06-03 10:59:07 UTC
Permalink
Documentation update about implementation details and requirements for Dynamic
RSS Configuration for Bonding.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 32 ++++++++++++++++++--
1 file changed, 30 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
index 03baf90..7f06cbe 100644
--- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
+++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
@@ -1,5 +1,5 @@
.. BSD LICENSE
- Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+ Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
All rights reserved.

Redistribution and use in source and binary forms, with or without
@@ -173,7 +173,27 @@ After a slave device is added to a bonded device slave is stopped using
``rte_eth_dev_stop`` and then reconfigured using ``rte_eth_dev_configure``
the RX and TX queues are also reconfigured using ``rte_eth_tx_queue_setup`` /
``rte_eth_rx_queue_setup`` with the parameters use to configure the bonding
-device.
+device. If RSS is enabled for bonding device, this mode is also enabled on new
+slave and configured as well.
+
+Setting up multi-queue mode for bonding device to RSS, makes it fully
+RSS-capable, so all slaves are synchronized with its configuration. This mode is
+intended to provide RSS configuration on slaves transparent for client
+application implementation.
+
+Bonding device stores its own version of RSS settings i.e. RETA, RSS hash
+function and RSS key, used to set up its slaves. That let to define the meaning
+of RSS configuration of bonding device as desired configuration of whole bonding
+(as one unit), without pointing any of slave inside. It is required to ensure
+consistency and made it more errorproof.
+
+RSS hash function set for bonding device, is a maximal set of RSS hash functions
+supported by all bonded slaves. RETA size is a GCD of all its RETA's sizes, so
+it can be easily used as a pattern providing expected behavior, even if slave
+RETAs' sizes are different. RSS key is always 40 bytes long.
+
+All settings are managed through the bonding port API and always are propagated
+in one direction (from bonding to slaves).

Link Status Change Interrupts / Polling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -207,6 +227,14 @@ these parameters.
A bonding device must have a minimum of one slave before the bonding device
itself can be started.

+To use a bonding device dynamic RSS configuration feature effectively, it is
+also required, that all slaves should be RSS-capable and support, at least one
+common hash function available for each of them.
+
+To prevent inconsistency on how slaves process packets, once a device is added
+to a bonding device, RSS configuration should be managed through the bonding
+device API, and not directly on the slave.
+
Like all other PMD, all functions exported by a PMD are lock-free functions
that are assumed not to be invoked in parallel on different logical cores to
work on the same target object.
--
1.7.9.5
Xu, HuilongX
2015-06-12 05:36:00 UTC
Permalink
Tested-by: huilong xu <***@intel.com>
- Tested Commit: 1a1109404e702d3ad1ccc1033df55c59bec1f89a + PATCH
- OS: Linux dpdk-fedora20 3.11.10-301.fc20.x86_64
- GCC: gcc version 4.8.3 20140624 (Red Hat 4.8.3-1) (GCC)
- CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
- NIC: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb]
- NIC: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ [8086:1583]
- Default x86_64-native-linuxapp-gcc configuration
- Total 4 cases, 2 passed, 2 failed. Niantic NIC case all passed, but Fortville NIC case all failed.


First case:
This is a unit test case, not need NIC
1. build dpdk driver and insmod driver
2. set 1024*2M hugepage
3. compile test app in app/test
4. run test
./test -c f -n 4
5. exec dynamic rss confif unit test
link_bonding_rssconf_autotest
6. print "test ok"
7. this case passed
Second case:
This is a function test case, used Fortville NIC(8086:1583)
1. build dpdk driver and insmod driver
2. bind dpdk driver to Fortville nic
3. set 1024*2M hugepage
4. run test pmd
./x86_64-native-linuxapp-gcc/app/testpmd -c 0xff -n 4 -- -i --txqflags=0x2 --mbcache=128 --burst=32 --txfreet=32 --rxfreet=64 --rxq=4 --txq=4
5. exec testpmd cmd
a) create bonded device 0 0
b) add bonding slave 0 3
c) add bonding slave 1 3
d) port start 3
port can start, and link stats is down, so this case failed.
Thirdly case:
This is a function test case, used Fortville NIC(8086:1583)
1. build dpdk driver and insmod driver
2. bind dpdk driver to Fortville nic
3. set 1024*2M hugepage
4. run test pmd
./x86_64-native-linuxapp-gcc/app/testpmd -c 0xff -n 4 -- -i --txqflags=0x2 --mbcache=128 --burst=32 --txfreet=32 --rxfreet=64 --rxq=4 --txq=4
5. exec testpmd cmd
a) create bonded device 0 0
b) add bonding slave 0 3
c) add bonding slave 1 3
d) port config all rss ip
e) show port 3 rss-hash
printf:
RSS functions:
ipv4-frag ipv4-other ipv6-frag ipv6-other
f) show port 0 rss-hash
printf:
RSS disabled
Slave rss not enable, so this case failed
Fourthly case:
This is a function test case, used Niantic NIC(8086:10fb)
1. build dpdk driver and insmod driver
2. bind dpdk driver to Fortville nic
3. set 1024*2M hugepage
4. run test pmd
./x86_64-native-linuxapp-gcc/app/testpmd -c 0xff -n 4 -- -i --txqflags=0x2 --mbcache=128 --burst=32 --txfreet=32 --rxfreet=64 --rxq=4 --txq=4
5. exec testpmd cmd
a) create bonded device 0 0
b) add bonding slave 0 3
c) add bonding slave 1 3
d) port start 3
e) port stop all
f) set verbose 8
g) set fwd rxonly
h) set stat_qmap 3 0 0
i) set stat_qmap 3 1 1
j) set stat_qmap 3 1 1
k) set stat_qmap 3 1 1
l) port config all rss ip
m) port start all
n)start
6. send 50 ip packages to salve 0 by ixia, the package config as below
a) dst mac: bond device (port 3) mac address.
b) src mac: 00:00:00:12:34:56
c) package type:0800
e) dst ip: 192.168.1.1
f) src ip: from 192.168.1.2 to 192.168.1.51, one package, this ip add 1
7. stop
Port 3 queue 0 received 9 packages
Port 3 queue 1 received 9 packages
Port 3 queue 2 received 16 packages
Port 3 queue 3 received 16 packages
8. send 50 ip packages to slave 1 by ixia, the package config as same
9. stop and check port 3 received packages again
Form slave 0 and slave 1 the rss are some, the test passed
-----Original Message-----
Sent: Wednesday, June 03, 2015 6:59 PM
Subject: [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding
OVERVIEW
--------
1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding
device fully RSS-capable, so all slaves are synchronized with its configuration.
This mode is intended to provide RSS configuration as known from "dynamic RSS
configuration for one port" and made slaves transparent for client application
implementation.
2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not
synchronized. That provides an ability to configure them manually. This mode may
be useful when application wants to manage RSS in an unusual way and the
consistency of RSS configuration for slaves isn't required.
Turning on/off RSS mode for slaves when bonding is started is not possible.
Other RSS configuration is propagated over slaves, when bonding device API is
used to do it.
bond: dynamic rss configuration
ring: dynamic rss configuration
test: dynamic rss configuration
bond: queue stats mapping
ring: queue stats mapping set dummy implementation
examples: dynamic rss configuration for bonding
doc: fixed spellings and typos
doc: dynamic rss configuration for bonding
app/test/Makefile | 1 +
app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 40 +-
drivers/net/bonding/rte_eth_bond_api.c | 22 +
drivers/net/bonding/rte_eth_bond_pmd.c | 222 ++++-
drivers/net/bonding/rte_eth_bond_private.h | 11 +
drivers/net/ring/rte_eth_ring.c | 133 ++-
examples/bond_rss/Makefile | 59 ++
examples/bond_rss/bondrss.c | 293 +++++++
examples/bond_rss/bondrss.h | 163 ++++
examples/bond_rss/config.c | 251 ++++++
examples/bond_rss/ui.c | 915
++++++++++++++++++++
12 files changed, 2759 insertions(+), 25 deletions(-)
create mode 100644 app/test/test_link_bonding_rssconf.c
create mode 100644 examples/bond_rss/Makefile
create mode 100644 examples/bond_rss/bondrss.c
create mode 100644 examples/bond_rss/bondrss.h
create mode 100644 examples/bond_rss/config.c
create mode 100644 examples/bond_rss/ui.c
--
1.7.9.5
Kulasek, TomaszX
2015-06-25 09:20:21 UTC
Permalink
There's a bug in bonding itself, which prevents the bonding, made of Fortville NICs, start and is not related to Dynamic RSS Configuration.

This problem solves separate patch "bond: fix check initial link status of slave".

-----Original Message-----
From: Xu, HuilongX
Sent: Friday, June 12, 2015 07:36
To: Kulasek, TomaszX; ***@dpdk.org
Subject: RE: [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding

Tested-by: huilong xu <***@intel.com>
- Tested Commit: 1a1109404e702d3ad1ccc1033df55c59bec1f89a + PATCH
- OS: Linux dpdk-fedora20 3.11.10-301.fc20.x86_64
- GCC: gcc version 4.8.3 20140624 (Red Hat 4.8.3-1) (GCC)
- CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
- NIC: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection [8086:10fb]
- NIC: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ [8086:1583]
- Default x86_64-native-linuxapp-gcc configuration
- Total 4 cases, 2 passed, 2 failed. Niantic NIC case all passed, but Fortville NIC case all failed.


First case:
This is a unit test case, not need NIC
1. build dpdk driver and insmod driver
2. set 1024*2M hugepage
3. compile test app in app/test
4. run test
./test -c f -n 4
5. exec dynamic rss confif unit test
link_bonding_rssconf_autotest
6. print "test ok"
7. this case passed
Second case:
This is a function test case, used Fortville NIC(8086:1583)
1. build dpdk driver and insmod driver
2. bind dpdk driver to Fortville nic
3. set 1024*2M hugepage
4. run test pmd
./x86_64-native-linuxapp-gcc/app/testpmd -c 0xff -n 4 -- -i --txqflags=0x2 --mbcache=128 --burst=32 --txfreet=32 --rxfreet=64 --rxq=4 --txq=4
5. exec testpmd cmd
a) create bonded device 0 0
b) add bonding slave 0 3
c) add bonding slave 1 3
d) port start 3
port can start, and link stats is down, so this case failed.
Thirdly case:
This is a function test case, used Fortville NIC(8086:1583)
1. build dpdk driver and insmod driver
2. bind dpdk driver to Fortville nic
3. set 1024*2M hugepage
4. run test pmd
./x86_64-native-linuxapp-gcc/app/testpmd -c 0xff -n 4 -- -i --txqflags=0x2 --mbcache=128 --burst=32 --txfreet=32 --rxfreet=64 --rxq=4 --txq=4
5. exec testpmd cmd
a) create bonded device 0 0
b) add bonding slave 0 3
c) add bonding slave 1 3
d) port config all rss ip
e) show port 3 rss-hash
printf:
RSS functions:
ipv4-frag ipv4-other ipv6-frag ipv6-other
f) show port 0 rss-hash
printf:
RSS disabled
Slave rss not enable, so this case failed
Fourthly case:
This is a function test case, used Niantic NIC(8086:10fb)
1. build dpdk driver and insmod driver
2. bind dpdk driver to Fortville nic
3. set 1024*2M hugepage
4. run test pmd
./x86_64-native-linuxapp-gcc/app/testpmd -c 0xff -n 4 -- -i --txqflags=0x2 --mbcache=128 --burst=32 --txfreet=32 --rxfreet=64 --rxq=4 --txq=4
5. exec testpmd cmd
a) create bonded device 0 0
b) add bonding slave 0 3
c) add bonding slave 1 3
d) port start 3
e) port stop all
f) set verbose 8
g) set fwd rxonly
h) set stat_qmap 3 0 0
i) set stat_qmap 3 1 1
j) set stat_qmap 3 1 1
k) set stat_qmap 3 1 1
l) port config all rss ip
m) port start all
n)start
6. send 50 ip packages to salve 0 by ixia, the package config as below
a) dst mac: bond device (port 3) mac address.
b) src mac: 00:00:00:12:34:56
c) package type:0800
e) dst ip: 192.168.1.1
f) src ip: from 192.168.1.2 to 192.168.1.51, one package, this ip add 1
7. stop
Port 3 queue 0 received 9 packages
Port 3 queue 1 received 9 packages
Port 3 queue 2 received 16 packages
Port 3 queue 3 received 16 packages
8. send 50 ip packages to slave 1 by ixia, the package config as same
9. stop and check port 3 received packages again
Form slave 0 and slave 1 the rss are some, the test passed
-----Original Message-----
Sent: Wednesday, June 03, 2015 6:59 PM
Subject: [dpdk-dev] [PATCH 0/8] Dynamic RSS Configuration for Bonding
OVERVIEW
--------
1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes
bonding device fully RSS-capable, so all slaves are synchronized with
its configuration.
This mode is intended to provide RSS configuration as known from
"dynamic RSS configuration for one port" and made slaves transparent
for client application implementation.
2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves
are not synchronized. That provides an ability to configure them
manually. This mode may be useful when application wants to manage RSS
in an unusual way and the consistency of RSS configuration for slaves
isn't required.
Turning on/off RSS mode for slaves when bonding is started is not possible.
Other RSS configuration is propagated over slaves, when bonding device
API is used to do it.
bond: dynamic rss configuration
ring: dynamic rss configuration
test: dynamic rss configuration
bond: queue stats mapping
ring: queue stats mapping set dummy implementation
examples: dynamic rss configuration for bonding
doc: fixed spellings and typos
doc: dynamic rss configuration for bonding
app/test/Makefile | 1 +
app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 40 +-
drivers/net/bonding/rte_eth_bond_api.c | 22 +
drivers/net/bonding/rte_eth_bond_pmd.c | 222 ++++-
drivers/net/bonding/rte_eth_bond_private.h | 11 +
drivers/net/ring/rte_eth_ring.c | 133 ++-
examples/bond_rss/Makefile | 59 ++
examples/bond_rss/bondrss.c | 293 +++++++
examples/bond_rss/bondrss.h | 163 ++++
examples/bond_rss/config.c | 251 ++++++
examples/bond_rss/ui.c | 915
++++++++++++++++++++
12 files changed, 2759 insertions(+), 25 deletions(-) create mode
100644 app/test/test_link_bonding_rssconf.c
create mode 100644 examples/bond_rss/Makefile create mode 100644
examples/bond_rss/bondrss.c create mode 100644
examples/bond_rss/bondrss.h create mode 100644
examples/bond_rss/config.c create mode 100644 examples/bond_rss/ui.c
--
1.7.9.5
Tomasz Kulasek
2015-06-19 14:13:12 UTC
Permalink
OVERVIEW
--------
1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding
device fully RSS-capable, so all slaves are synchronized with its configuration.
This mode is intended to provide RSS configuration as known from "dynamic RSS
configuration for one port" and made slaves transparent for client application
implementation.

2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not
synchronized. That provides an ability to configure them manually. This mode may
be useful when application wants to manage RSS in an unusual way and the
consistency of RSS configuration for slaves isn't required.

Turning on/off RSS mode for slaves when bonding is started is not possible.
Other RSS configuration is propagated over slaves, when bonding device API is
used to do it.

v2 changes:
- added support for keys other than 40 bytes long,
- now, if RSS key is not set for bonding, it is not set also for slaves,
- fix - full initial RSS configuration before any slave is added was not
possible due to the initially zeroed flow_type_rss_offloads for bonding,
- fix - changed error to warning when slave is synchronizing due to the
bonding's initial configuration (to allow use slaves' drivers not supporting
dynamic RSS configuration in bonding),
- some code cleanups,
- updated documentation,

Tomasz Kulasek (8):
bond: rss dynamic configuration
ring: dynamic rss configuration
test: dynamic rss configuration
bond: queue stats mapping
ring: queue stats mapping set dummy implementation
examples: dynamic rss configuration for bonding
doc: fixed spellings and typos
doc: dynamic rss configuration for bonding

app/test/Makefile | 1 +
app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 42 +-
drivers/net/bonding/rte_eth_bond_api.c | 27 +
drivers/net/bonding/rte_eth_bond_pmd.c | 239 ++++-
drivers/net/bonding/rte_eth_bond_private.h | 12 +
drivers/net/ring/rte_eth_ring.c | 133 ++-
examples/bond_rss/Makefile | 59 ++
examples/bond_rss/bondrss.c | 293 +++++++
examples/bond_rss/bondrss.h | 163 ++++
examples/bond_rss/config.c | 251 ++++++
examples/bond_rss/ui.c | 920 ++++++++++++++++++++
12 files changed, 2789 insertions(+), 25 deletions(-)
create mode 100644 app/test/test_link_bonding_rssconf.c
create mode 100644 examples/bond_rss/Makefile
create mode 100644 examples/bond_rss/bondrss.c
create mode 100644 examples/bond_rss/bondrss.h
create mode 100644 examples/bond_rss/config.c
create mode 100644 examples/bond_rss/ui.c
--
1.7.9.5
Tomasz Kulasek
2015-06-19 14:13:13 UTC
Permalink
Bonding device implements independent management of RSS settings. It stores its
own copies of settings i.e. RETA, RSS hash function and RSS key. It’s required
to ensure consistency.

1) RSS hash function set for bonding device is maximal set of RSS hash functions
supported by all bonded devices. That mean, to have RSS support for bonding, all
slaves should be RSS-capable.

2) RSS key is propagated over the slaves "as is".

3) RETA for bonding is an internal table managed by bonding API, and is used as
a pattern to set up slaves. Its size is GCD of all RETA sizes, so it can be
easily used as a pattern providing expected behavior, even if slaves RETA sizes
are different.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/bonding/rte_eth_bond_api.c | 27 ++++
drivers/net/bonding/rte_eth_bond_pmd.c | 209 ++++++++++++++++++++++++++--
drivers/net/bonding/rte_eth_bond_private.h | 12 ++
3 files changed, 234 insertions(+), 14 deletions(-)

diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c
index d4caa83..1d8083d 100644
--- a/drivers/net/bonding/rte_eth_bond_api.c
+++ b/drivers/net/bonding/rte_eth_bond_api.c
@@ -302,6 +302,9 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id)
internals->rx_offload_capa = 0;
internals->tx_offload_capa = 0;

+ /* Initially allow to choose any offload type */
+ internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK;
+
memset(internals->active_slaves, 0, sizeof(internals->active_slaves));
memset(internals->slaves, 0, sizeof(internals->slaves));

@@ -365,6 +368,11 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)

rte_eth_dev_info_get(slave_port_id, &dev_info);

+ /* We need to store slaves reta_size to be able to synchronize RETA for all
+ * slave devices even if its sizes are different.
+ */
+ internals->slaves[internals->slave_count].reta_size = dev_info.reta_size;
+
if (internals->slave_count < 1) {
/* if MAC is not user defined then use MAC of first slave add to
* bonded device */
@@ -378,9 +386,16 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
/* Make primary slave */
internals->primary_port = slave_port_id;

+ /* Inherit queues settings from first slave */
+ internals->nb_rx_queues = slave_eth_dev->data->nb_rx_queues;
+ internals->nb_tx_queues = slave_eth_dev->data->nb_tx_queues;
+
+ internals->reta_size = dev_info.reta_size;
+
/* Take the first dev's offload capabilities */
internals->rx_offload_capa = dev_info.rx_offload_capa;
internals->tx_offload_capa = dev_info.tx_offload_capa;
+ internals->flow_type_rss_offloads = dev_info.flow_type_rss_offloads;

} else {
/* Check slave link properties are supported if props are set,
@@ -399,8 +414,18 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
}
internals->rx_offload_capa &= dev_info.rx_offload_capa;
internals->tx_offload_capa &= dev_info.tx_offload_capa;
+ internals->flow_type_rss_offloads &= dev_info.flow_type_rss_offloads;
+
+ /* RETA size is GCD of all slaves RETA sizes,
+ * so, if all sizes will be the power of 2, the lower one is GCD */
+ if (internals->reta_size > dev_info.reta_size)
+ internals->reta_size = dev_info.reta_size;
+
}

+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf &=
+ internals->flow_type_rss_offloads;
+
internals->slave_count++;

/* Update all slave devices MACs*/
@@ -527,6 +552,8 @@ __eth_bond_slave_remove_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
if (internals->slave_count == 0) {
internals->rx_offload_capa = 0;
internals->tx_offload_capa = 0;
+ internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK;
+ internals->reta_size = 0;
}
return 0;
}
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 8bad2e1..5de2f30 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -1306,6 +1306,22 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev,
if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC)
slave_eth_dev->data->dev_conf.intr_conf.lsc = 1;

+ /* If RSS is enabled for bonding, try to enable it for slaves */
+ if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) {
+ if (bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len != 0) {
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len =
+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len;
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key =
+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key;
+ }
+ else
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL;
+
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ slave_eth_dev->data->dev_conf.rxmode.mq_mode |= ETH_MQ_RX_RSS;
+ }
+
/* Configure device */
errval = rte_eth_dev_configure(slave_eth_dev->data->port_id,
bonded_eth_dev->data->nb_rx_queues,
@@ -1357,6 +1373,31 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev,
return -1;
}

+ /* If RSS is enabled for bonding, synchronize RETA */
+ if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) {
+ int i;
+ struct bond_dev_private *internals;
+ internals = bonded_eth_dev->data->dev_private;
+
+ for (i = 0; i < internals->slave_count; i++) {
+ if (internals->slaves[i].port_id == slave_eth_dev->data->port_id) {
+ errval = rte_eth_dev_rss_reta_update(
+ slave_eth_dev->data->port_id,
+ &(internals->reta_conf[0]),
+ internals->slaves[i].reta_size);
+ if (errval != 0) {
+ RTE_LOG(WARNING, PMD,
+ "Can't update slave's RSS configuration "
+ "(rte_eth_dev_rss_reta_update on slave port fails: "
+ "port=%d, err %d). RSS Configuration for bonding "
+ "may be inconsistent.\n",
+ slave_eth_dev->data->port_id, errval);
+ }
+ break;
+ }
+ }
+ }
+
return 0;
}

@@ -1569,6 +1610,9 @@ bond_ethdev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)

dev_info->rx_offload_capa = internals->rx_offload_capa;
dev_info->tx_offload_capa = internals->tx_offload_capa;
+ dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads;
+
+ dev_info->reta_size = internals->reta_size;
}

static int
@@ -1950,21 +1994,134 @@ bond_ethdev_lsc_event_callback(uint8_t port_id, enum rte_eth_event_type type,
}
}

+static int
+bond_ethdev_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf,
+ uint16_t reta_size)
+{
+ unsigned i, j;
+ int result = 0;
+ int slave_reta_size;
+ unsigned reta_count;
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ if (reta_size != internals->reta_size)
+ return -EINVAL;
+
+ /* Copy RETA table */
+ reta_count = reta_size / RTE_RETA_GROUP_SIZE;
+
+ for (i = 0; i < reta_count; i++) {
+ internals->reta_conf[i].mask = reta_conf[i].mask;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ internals->reta_conf[i].reta[j] = reta_conf[i].reta[j];
+ }
+
+ /* Fill rest of array */
+ for (; i < RTE_DIM(internals->reta_conf); i += reta_count)
+ memcpy(&(internals->reta_conf[i]), &(internals->reta_conf[0]),
+ sizeof(internals->reta_conf[0]) * reta_count);
+
+ /* Propagate RETA over slaves */
+ for (i = 0; i < internals->slave_count; i++) {
+ slave_reta_size = internals->slaves[i].reta_size;
+ result = rte_eth_dev_rss_reta_update(internals->slaves[i].port_id,
+ &(internals->reta_conf[0]), slave_reta_size);
+ if (result < 0)
+ return result;
+ }
+
+ return 0;
+}
+
+static int
+bond_ethdev_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf,
+ uint16_t reta_size)
+{
+ int i, j;
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ if (reta_size != internals->reta_size)
+ return -EINVAL;
+
+ /* Copy RETA table */
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++)
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ reta_conf[i].reta[j] = internals->reta_conf[i].reta[j];
+
+ return 0;
+}
+
+static int
+bond_ethdev_rss_hash_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ int i, result = 0;
+ struct bond_dev_private *internals = dev->data->dev_private;
+ struct rte_eth_rss_conf bond_rss_conf;
+
+ memcpy(&bond_rss_conf, rss_conf, sizeof(struct rte_eth_rss_conf));
+
+ bond_rss_conf.rss_hf &= internals->flow_type_rss_offloads;
+
+ if (bond_rss_conf.rss_hf != 0)
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = bond_rss_conf.rss_hf;
+
+ if (bond_rss_conf.rss_key != NULL && bond_rss_conf.rss_key_len <
+ sizeof(internals->rss_key)) {
+ if (bond_rss_conf.rss_key_len == 0)
+ bond_rss_conf.rss_key_len = 40;
+ internals->rss_key_len = bond_rss_conf.rss_key_len;
+ memcpy(internals->rss_key, bond_rss_conf.rss_key,
+ internals->rss_key_len);
+ }
+
+ for (i = 0; i < internals->slave_count; i++) {
+ result = rte_eth_dev_rss_hash_update(internals->slaves[i].port_id,
+ &bond_rss_conf);
+ if (result < 0)
+ return result;
+ }
+
+ return 0;
+}
+
+static int
+bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ rss_conf->rss_key_len = internals->rss_key_len;
+ if (rss_conf->rss_key != NULL)
+ memcpy(rss_conf->rss_key, internals->rss_key, internals->rss_key_len);
+
+ return 0;
+}
+
struct eth_dev_ops default_dev_ops = {
- .dev_start = bond_ethdev_start,
- .dev_stop = bond_ethdev_stop,
- .dev_close = bond_ethdev_close,
- .dev_configure = bond_ethdev_configure,
- .dev_infos_get = bond_ethdev_info,
- .rx_queue_setup = bond_ethdev_rx_queue_setup,
- .tx_queue_setup = bond_ethdev_tx_queue_setup,
- .rx_queue_release = bond_ethdev_rx_queue_release,
- .tx_queue_release = bond_ethdev_tx_queue_release,
- .link_update = bond_ethdev_link_update,
- .stats_get = bond_ethdev_stats_get,
- .stats_reset = bond_ethdev_stats_reset,
- .promiscuous_enable = bond_ethdev_promiscuous_enable,
- .promiscuous_disable = bond_ethdev_promiscuous_disable
+ .dev_start = bond_ethdev_start,
+ .dev_stop = bond_ethdev_stop,
+ .dev_close = bond_ethdev_close,
+ .dev_configure = bond_ethdev_configure,
+ .dev_infos_get = bond_ethdev_info,
+ .rx_queue_setup = bond_ethdev_rx_queue_setup,
+ .tx_queue_setup = bond_ethdev_tx_queue_setup,
+ .rx_queue_release = bond_ethdev_rx_queue_release,
+ .tx_queue_release = bond_ethdev_tx_queue_release,
+ .link_update = bond_ethdev_link_update,
+ .stats_get = bond_ethdev_stats_get,
+ .stats_reset = bond_ethdev_stats_reset,
+ .promiscuous_enable = bond_ethdev_promiscuous_enable,
+ .promiscuous_disable = bond_ethdev_promiscuous_disable,
+ .reta_update = bond_ethdev_rss_reta_update,
+ .reta_query = bond_ethdev_rss_reta_query,
+ .rss_hash_update = bond_ethdev_rss_hash_update,
+ .rss_hash_conf_get = bond_ethdev_rss_hash_conf_get
};

static int
@@ -2045,6 +2202,30 @@ bond_ethdev_configure(struct rte_eth_dev *dev)
int arg_count;
uint8_t port_id = dev - rte_eth_devices;

+ static const uint8_t default_rss_key[40] = {
+ 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D,
+ 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+ 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B,
+ 0xBE, 0xAC, 0x01, 0xFA
+ };
+
+ unsigned i, j;
+
+ /* If RSS is enabled, fill table and key with default values */
+ if (dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) {
+
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = internals->rss_key;
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 0;
+ memcpy(internals->rss_key, default_rss_key, 40);
+
+ for (i = 0; i < RTE_DIM(internals->reta_conf); i++) {
+ internals->reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ internals->reta_conf[i].reta[j] = j % dev->data->nb_rx_queues;
+ }
+
+ }
+
/*
* if no kvlist, it means that this bonded device has been created
* through the bonding api.
diff --git a/drivers/net/bonding/rte_eth_bond_private.h b/drivers/net/bonding/rte_eth_bond_private.h
index 45e5c65..f2ce94f 100644
--- a/drivers/net/bonding/rte_eth_bond_private.h
+++ b/drivers/net/bonding/rte_eth_bond_private.h
@@ -103,6 +103,8 @@ struct bond_slave_details {
uint8_t last_link_status;
/**< Port Id of slave eth_dev */
struct ether_addr persisted_mac_addr;
+
+ uint16_t reta_size;
};


@@ -155,6 +157,16 @@ struct bond_dev_private {
uint32_t rx_offload_capa; /** Rx offload capability */
uint32_t tx_offload_capa; /** Tx offload capability */

+ /** Bit mask of RSS offloads, the bit offset also means flow type */
+ uint64_t flow_type_rss_offloads;
+
+ uint16_t reta_size;
+ struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_512 /
+ RTE_RETA_GROUP_SIZE];
+
+ uint8_t rss_key[52]; /**< 52-byte hash key buffer. */
+ uint8_t rss_key_len; /**< hash key length in bytes. */
+
struct rte_kvargs *kvlist;
uint8_t slave_update_idx;
};
--
1.7.9.5
Tomasz Kulasek
2015-06-19 14:13:14 UTC
Permalink
This implementation allows to set and read RSS configuration for ring device,
and is used to validate right values propagation over the slaves, in test units
for dynamic RSS configuration for bonding.

It have no impact on packet processing by ring device.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/ring/rte_eth_ring.c | 122 +++++++++++++++++++++++++++++++++++++--
1 file changed, 118 insertions(+), 4 deletions(-)

diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index 6832f01..d7e7d9c 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -39,6 +39,7 @@
#include <rte_string_fns.h>
#include <rte_dev.h>
#include <rte_kvargs.h>
+#include <rte_spinlock.h>

#define ETH_RING_NUMA_NODE_ACTION_ARG "nodeaction"
#define ETH_RING_ACTION_CREATE "CREATE"
@@ -66,6 +67,17 @@ struct pmd_internals {
struct ring_queue tx_ring_queues[RTE_PMD_RING_MAX_TX_RINGS];

struct ether_addr address;
+
+ /** Bit mask of RSS offloads, the bit offset also means flow type */
+ uint64_t flow_type_rss_offloads;
+
+ rte_spinlock_t rss_lock;
+
+ uint16_t reta_size;
+ struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_128 /
+ RTE_RETA_GROUP_SIZE];
+
+ uint8_t rss_key[40]; /**< 40-byte hash key. */
};


@@ -173,6 +185,8 @@ eth_dev_info(struct rte_eth_dev *dev,
dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues;
dev_info->min_rx_bufsize = 0;
dev_info->pci_dev = NULL;
+ dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads;
+ dev_info->reta_size = internals->reta_size;
}

static void
@@ -234,6 +248,93 @@ static int
eth_link_update(struct rte_eth_dev *dev __rte_unused,
int wait_to_complete __rte_unused) { return 0; }

+static int
+eth_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf,
+ uint16_t reta_size)
+{
+ int i, j;
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ if (reta_size != internal->reta_size)
+ return -EINVAL;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ /* Copy RETA table */
+ for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) {
+ internal->reta_conf[i].mask = reta_conf[i].mask;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ internal->reta_conf[i].reta[j] = reta_conf[i].reta[j];
+ }
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf,
+ uint16_t reta_size)
+{
+ int i, j;
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ if (reta_size != internal->reta_size) return -EINVAL;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ /* Copy RETA table */
+ for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) {
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ reta_conf[i].reta[j] = internal->reta_conf[i].reta[j];
+ }
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_hash_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0)
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ rss_conf->rss_hf & internal->flow_type_rss_offloads;
+
+ if (rss_conf->rss_key != NULL)
+ memcpy(internal->rss_key, rss_conf->rss_key, 40);
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ if (rss_conf->rss_key != NULL)
+ memcpy(rss_conf->rss_key, internal->rss_key, 40);
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
static const struct eth_dev_ops ops = {
.dev_start = eth_dev_start,
.dev_stop = eth_dev_stop,
@@ -250,6 +351,10 @@ static const struct eth_dev_ops ops = {
.stats_reset = eth_stats_reset,
.mac_addr_remove = eth_mac_addr_remove,
.mac_addr_add = eth_mac_addr_add,
+ .reta_update = eth_rss_reta_update,
+ .reta_query = eth_rss_reta_query,
+ .rss_hash_update = eth_rss_hash_update,
+ .rss_hash_conf_get = eth_rss_hash_conf_get
};

int
@@ -268,6 +373,13 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[],

unsigned i;

+ static const uint8_t default_rss_key[40] = {
+ 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D,
+ 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+ 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B,
+ 0xBE, 0xAC, 0x01, 0xFA
+ };
+
/* do some parameter checking */
if (rx_queues == NULL && nb_rx_queues > 0)
goto error;
@@ -316,12 +428,14 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[],

internals->nb_rx_queues = nb_rx_queues;
internals->nb_tx_queues = nb_tx_queues;
- for (i = 0; i < nb_rx_queues; i++) {
+ for (i = 0; i < nb_rx_queues; i++)
internals->rx_ring_queues[i].rng = rx_queues[i];
- }
- for (i = 0; i < nb_tx_queues; i++) {
+ for (i = 0; i < nb_tx_queues; i++)
internals->tx_ring_queues[i].rng = tx_queues[i];
- }
+ internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK;
+ internals->reta_size = RTE_DIM(internals->reta_conf) * RTE_RETA_GROUP_SIZE;
+
+ memcpy(internals->rss_key, default_rss_key, 40);

eth_drv->pci_drv.name = ring_ethdev_driver_name;
eth_drv->pci_drv.id_table = id_table;
--
1.7.9.5
Tomasz Kulasek
2015-06-19 14:13:15 UTC
Permalink
This test module uses ring device to check right RSS configuration propagation.

1) Propagation test
a) Set RSS hash function for bonding, fetch RSS hash function from bonded
slave, check if same. Do it for all slaves.
b) Repeat above for RSS key and RETA.

2)Dynamic adding slave to the bonding
a) Remove slave from bonding.
b) Change its configuration to other than bonding.
c) Add slave to the started bonding device.
d) Check if slaves configuration changed.

3) Repeat point 1) and 2) with RSS multi-queue mode enabled/disabled for bonding
port.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
app/test/Makefile | 1 +
app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++++++++++++++++++++++
2 files changed, 675 insertions(+)
create mode 100644 app/test/test_link_bonding_rssconf.c

diff --git a/app/test/Makefile b/app/test/Makefile
index 5cf8296..64e5fa7 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -135,6 +135,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_ACL) += test_acl.c
ifeq ($(CONFIG_RTE_LIBRTE_PMD_RING),y)
SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_mode4.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_rssconf.c
endif

SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring.c
diff --git a/app/test/test_link_bonding_rssconf.c b/app/test/test_link_bonding_rssconf.c
new file mode 100644
index 0000000..c0be755
--- /dev/null
+++ b/app/test/test_link_bonding_rssconf.c
@@ -0,0 +1,674 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <errno.h>
+#include <rte_cycles.h>
+#include <sys/queue.h>
+
+#include <rte_byteorder.h>
+#include <rte_common.h>
+#include <rte_debug.h>
+#include <rte_ethdev.h>
+#include <rte_log.h>
+#include <rte_lcore.h>
+#include <rte_memory.h>
+
+#include <rte_string_fns.h>
+
+#include <rte_eth_ring.h>
+#include <rte_errno.h>
+#include <rte_eth_bond.h>
+
+#include "test.h"
+
+#define SLAVE_COUNT (4)
+
+#define RXTX_RING_SIZE 1024
+#define RXTX_QUEUE_COUNT 4
+
+#define BONDED_DEV_NAME ("rssconf_bond_dev")
+
+#define SLAVE_DEV_NAME_FMT ("rssconf_slave%d")
+#define SLAVE_RXTX_QUEUE_FMT ("rssconf_slave%d_q%d")
+
+#define NUM_MBUFS 8191
+#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)
+#define MBUF_CACHE_SIZE 250
+#define BURST_SIZE 32
+
+#define INVALID_SOCKET_ID (-1)
+#define INVALID_PORT_ID (0xFF)
+#define INVALID_BONDING_MODE (-1)
+
+struct slave_conf {
+ uint8_t port_id;
+ struct rte_eth_dev_info dev_info;
+
+ struct rte_eth_rss_conf rss_conf;
+ uint8_t rss_key[40];
+ struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE];
+
+ uint8_t is_slave;
+ struct rte_ring *rxtx_queue[RXTX_QUEUE_COUNT];
+};
+
+struct link_bonding_rssconf_unittest_params {
+ uint8_t bond_port_id;
+ struct rte_eth_dev_info bond_dev_info;
+ struct rte_eth_rss_reta_entry64 bond_reta_conf[512 / RTE_RETA_GROUP_SIZE];
+ struct slave_conf slave_ports[SLAVE_COUNT];
+
+ struct rte_mempool *mbuf_pool;
+};
+
+static struct link_bonding_rssconf_unittest_params test_params = {
+ .bond_port_id = INVALID_PORT_ID,
+ .slave_ports = {
+ [0 ... SLAVE_COUNT - 1] = { .port_id = INVALID_PORT_ID, .is_slave = 0}
+ },
+ .mbuf_pool = NULL,
+};
+
+/**
+ * Default port configuration with RSS turned off
+ */
+static struct rte_eth_conf default_pmd_conf = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_NONE,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled */
+ .hw_ip_checksum = 0, /**< IP checksum offload enabled */
+ .hw_vlan_filter = 0, /**< VLAN filtering disabled */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled */
+ .hw_strip_crc = 0, /**< CRC stripped by hardware */
+ },
+ .txmode = {
+ .mq_mode = ETH_MQ_TX_NONE,
+ },
+ .lpbk_mode = 0,
+};
+
+static struct rte_eth_conf rss_pmd_conf = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_RSS,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled */
+ .hw_ip_checksum = 0, /**< IP checksum offload enabled */
+ .hw_vlan_filter = 0, /**< VLAN filtering disabled */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled */
+ .hw_strip_crc = 0, /**< CRC stripped by hardware */
+ },
+ .txmode = {
+ .mq_mode = ETH_MQ_TX_NONE,
+ },
+ .rx_adv_conf = {
+ .rss_conf = {
+ .rss_key = NULL,
+ .rss_hf = ETH_RSS_IPV6,
+ },
+ },
+ .lpbk_mode = 0,
+};
+
+#define FOR_EACH(_i, _item, _array, _size) \
+ for (_i = 0, _item = &_array[0]; _i < _size && (_item = &_array[_i]); _i++)
+
+/* Macro for iterating over every port that can be used as a slave
+ * in this test.
+ * _i variable used as an index in test_params->slave_ports
+ * _slave pointer to &test_params->slave_ports[_idx]
+ */
+#define FOR_EACH_PORT(_i, _port) \
+ FOR_EACH(_i, _port, test_params.slave_ports, \
+ RTE_DIM(test_params.slave_ports))
+
+static int
+configure_ethdev(uint8_t port_id, struct rte_eth_conf *eth_conf, uint8_t start)
+{
+ int rxq, txq;
+
+ TEST_ASSERT(rte_eth_dev_configure(port_id, RXTX_QUEUE_COUNT,
+ RXTX_QUEUE_COUNT, eth_conf) == 0, "Failed to configure device %u",
+ port_id);
+
+ for (rxq = 0; rxq < RXTX_QUEUE_COUNT; rxq++) {
+ TEST_ASSERT(rte_eth_rx_queue_setup(port_id, rxq, RXTX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id), NULL,
+ test_params.mbuf_pool) == 0, "Failed to setup rx queue.");
+ }
+
+ for (txq = 0; txq < RXTX_QUEUE_COUNT; txq++) {
+ TEST_ASSERT(rte_eth_tx_queue_setup(port_id, txq, RXTX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id), NULL) == 0,
+ "Failed to setup tx queue.");
+ }
+
+ if (start) {
+ TEST_ASSERT(rte_eth_dev_start(port_id) == 0,
+ "Failed to start device (%d).", port_id);
+ }
+
+ return 0;
+}
+
+/**
+ * Remove all slaves from bonding
+ */
+static int
+remove_slaves(void)
+{
+ unsigned n;
+ struct slave_conf *port;
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ if (port->is_slave) {
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(
+ test_params.bond_port_id, port->port_id),
+ "Cannot remove slave %d from bonding", port->port_id);
+ port->is_slave = 0;
+ }
+ }
+
+ return 0;
+}
+
+static int
+remove_slaves_and_stop_bonded_device(void)
+{
+ TEST_ASSERT_SUCCESS(remove_slaves(), "Removing slaves");
+ rte_eth_dev_stop(test_params.bond_port_id);
+ return TEST_SUCCESS;
+}
+
+/**
+ * Add all slaves to bonding
+ */
+static int
+bond_slaves(void)
+{
+ unsigned n;
+ struct slave_conf *port;
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ if (!port->is_slave) {
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id,
+ port->port_id), "Cannot attach slave %d to the bonding",
+ port->port_id);
+ port->is_slave = 1;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * Set all RETA values in port_id to value
+ */
+static int
+reta_set(uint8_t port_id, uint8_t value, int reta_size)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ int i, j;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) {
+ /* select all fields to set */
+ reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ reta_conf[i].reta[j] = value;
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
+
+/**
+ * Check if slaves RETA is synchronized with bonding port. Returns 1 if slave
+ * port is synced with bonding port.
+ */
+static int
+reta_check_synced(struct slave_conf *port)
+{
+ unsigned i;
+
+ for (i = 0; i < test_params.bond_dev_info.reta_size;
+ i++) {
+
+ int index = i / RTE_RETA_GROUP_SIZE;
+ int shift = i % RTE_RETA_GROUP_SIZE;
+
+ if (port->reta_conf[index].reta[shift] !=
+ test_params.bond_reta_conf[index].reta[shift])
+ return 0;
+
+ }
+
+ return 1;
+}
+
+/**
+ * Fetch bonding ports RETA
+ */
+static int
+bond_reta_fetch(void) {
+ unsigned j;
+
+ for (j = 0; j < test_params.bond_dev_info.reta_size / RTE_RETA_GROUP_SIZE;
+ j++)
+ test_params.bond_reta_conf[j].mask = ~0LL;
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(test_params.bond_port_id,
+ test_params.bond_reta_conf, test_params.bond_dev_info.reta_size),
+ "Cannot take bonding ports RSS configuration");
+ return 0;
+}
+
+/**
+ * Fetch slaves RETA
+ */
+static int
+slave_reta_fetch(struct slave_conf *port) {
+ unsigned j;
+
+ for (j = 0; j < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; j++)
+ port->reta_conf[j].mask = ~0LL;
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(port->port_id,
+ port->reta_conf, port->dev_info.reta_size),
+ "Cannot take bonding ports RSS configuration");
+ return 0;
+}
+
+/**
+ * Remove and add slave to check if slaves configuration is synced with
+ * the bonding ports values after adding new slave.
+ */
+static int
+slave_remove_and_add(void)
+{
+ struct slave_conf *port = &(test_params.slave_ports[0]);
+
+ /* 1. Remove first slave from bonding */
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(test_params.bond_port_id,
+ port->port_id), "Cannot remove slave #d from bonding");
+
+ /* 2. Change removed (ex-)slave and bonding configuration to different
+ * values
+ */
+ reta_set(test_params.bond_port_id, 1, test_params.bond_dev_info.reta_size);
+ bond_reta_fetch();
+
+ reta_set(port->port_id, 2, port->dev_info.reta_size);
+ slave_reta_fetch(port);
+
+ TEST_ASSERT(reta_check_synced(port) == 0,
+ "Removed slave didn't should be synchronized with bonding port");
+
+ /* 3. Add (ex-)slave and check if configuration changed*/
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id,
+ port->port_id), "Cannot add slave");
+
+ bond_reta_fetch();
+ slave_reta_fetch(port);
+
+ return reta_check_synced(port);
+}
+
+/**
+ * Test configuration propagation over slaves.
+ */
+static int
+test_propagate(void)
+{
+ unsigned i;
+ uint8_t n;
+ struct slave_conf *port;
+ uint8_t bond_rss_key[40];
+ struct rte_eth_rss_conf bond_rss_conf;
+
+ int retval = 0;
+ uint64_t rss_hf = 0;
+ uint64_t default_rss_hf = 0;
+
+ rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info);
+
+ /*
+ * Test hash function propagation
+ */
+ for (i = 0; i < sizeof(test_params.bond_dev_info.flow_type_rss_offloads)*8;
+ i++) {
+
+ rss_hf = test_params.bond_dev_info.flow_type_rss_offloads & (1<<i);
+ if (rss_hf) {
+ bond_rss_conf.rss_key = NULL;
+ bond_rss_conf.rss_hf = rss_hf;
+
+ retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id,
+ &bond_rss_conf);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set slaves hash function");
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ retval = rte_eth_dev_rss_hash_conf_get(port->port_id,
+ &port->rss_conf);
+ TEST_ASSERT_SUCCESS(retval,
+ "Cannot take slaves RSS configuration");
+
+ TEST_ASSERT(port->rss_conf.rss_hf == rss_hf,
+ "Hash function not propagated for slave %d",
+ port->port_id);
+ }
+
+ default_rss_hf = rss_hf;
+ }
+
+ }
+
+ /*
+ * Test key propagation
+ */
+ for (i = 1; i < 10; i++) {
+
+ /* Set all keys to zero */
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ memset(port->rss_conf.rss_key, 0, 40);
+ retval = rte_eth_dev_rss_hash_update(port->port_id,
+ &port->rss_conf);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RSS keys");
+ }
+
+ memset(bond_rss_key, i, sizeof(bond_rss_key));
+ bond_rss_conf.rss_hf = default_rss_hf,
+ bond_rss_conf.rss_key = bond_rss_key;
+ bond_rss_conf.rss_key_len = 40;
+
+ retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id,
+ &bond_rss_conf);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set bonded port RSS keys");
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ retval = rte_eth_dev_rss_hash_conf_get(port->port_id,
+ &(port->rss_conf));
+
+ TEST_ASSERT_SUCCESS(retval,
+ "Cannot take slaves RSS configuration");
+
+ /* compare keys */
+ retval = memcmp(port->rss_conf.rss_key, bond_rss_key,
+ sizeof(bond_rss_key));
+ TEST_ASSERT(retval == 0, "Key value not propagated for slave %d",
+ port->port_id);
+ }
+ }
+
+ /*
+ * Test RETA propagation
+ */
+ for (i = 0; i < RXTX_QUEUE_COUNT; i++) {
+
+ /* Set all keys to zero */
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ retval = reta_set(port->port_id, (i + 1) % RXTX_QUEUE_COUNT,
+ port->dev_info.reta_size);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RETA");
+ }
+
+ TEST_ASSERT_SUCCESS(reta_set(test_params.bond_port_id,
+ i % RXTX_QUEUE_COUNT, test_params.bond_dev_info.reta_size),
+ "Cannot set bonded port RETA");
+
+ bond_reta_fetch();
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ slave_reta_fetch(port);
+ TEST_ASSERT(reta_check_synced(port) == 1, "RETAs inconsistent");
+ }
+ }
+
+ return TEST_SUCCESS;
+}
+
+/**
+ * Test propagation logic, when RX_RSS mq_mode is turned on for bonding port
+ */
+static int
+test_rss(void)
+{
+ /**
+ * Configure bonding port in RSS mq mode
+ */
+ TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id,
+ &rss_pmd_conf, 0), "Failed to configure bonding device\n");
+
+ rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info);
+
+ TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed");
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id),
+ "Failed to start bonding port (%d).", test_params.bond_port_id);
+
+ TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed");
+
+ TEST_ASSERT(slave_remove_and_add() == 1, "New slave should be synced");
+
+ remove_slaves_and_stop_bonded_device();
+
+ return TEST_SUCCESS;
+}
+
+/**
+ * Test propagation logic, when RX_RSS mq_mode is turned off for bonding port
+ */
+static int
+test_rss_lazy(void)
+{
+ TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id,
+ &default_pmd_conf, 0), "Failed to configure bonding device\n");
+
+ rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info);
+
+ TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed");
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id),
+ "Failed to start bonding port (%d).", test_params.bond_port_id);
+
+ TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed");
+
+ TEST_ASSERT(slave_remove_and_add() == 0, "New slave shouldn't be synced");
+
+ remove_slaves_and_stop_bonded_device();
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_setup(void)
+{
+ unsigned i, n;
+ int retval;
+ char name[256];
+ char rng_name[RTE_RING_NAMESIZE];
+ struct slave_conf *port;
+
+ if (test_params.mbuf_pool == NULL) {
+
+ test_params.mbuf_pool = rte_mempool_create("MBUF_POOL", NUM_MBUFS *
+ SLAVE_COUNT, MBUF_SIZE, MBUF_CACHE_SIZE,
+ sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init,
+ NULL, rte_pktmbuf_init, NULL, rte_socket_id(), 0);
+
+ TEST_ASSERT(test_params.mbuf_pool != NULL,
+ "rte_mempool_create failed\n");
+ }
+
+ /* Create / initialize ring eth devs. */
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ snprintf(name, sizeof(name), SLAVE_DEV_NAME_FMT, n);
+
+ for (i = 0; i < RXTX_QUEUE_COUNT; i++) {
+ snprintf(rng_name, sizeof(rng_name), SLAVE_RXTX_QUEUE_FMT, n, i);
+
+ if (port->rxtx_queue[i] == NULL) {
+ port->rxtx_queue[i] = rte_ring_create(rng_name, RXTX_RING_SIZE,
+ 0, RING_F_SP_ENQ|RING_F_SC_DEQ);
+ TEST_ASSERT(port->rxtx_queue[i] != NULL,
+ "Failed to allocate rx ring '%s': %s", name,
+ rte_strerror(rte_errno));
+ }
+ }
+
+ if (rte_eth_from_rings(name, port->rxtx_queue, RXTX_QUEUE_COUNT,
+ port->rxtx_queue, RXTX_QUEUE_COUNT, 0)) {
+ TEST_ASSERT(retval >= 0,
+ "Failed to create ring ethdev '%s'\n", name);
+ }
+
+ port->port_id = rte_eth_dev_count() - 1;
+ port->rss_conf.rss_key = port->rss_key;
+ port->rss_conf.rss_key_len = 40;
+
+ retval = configure_ethdev(port->port_id, &default_pmd_conf, 1);
+ TEST_ASSERT_SUCCESS(retval, "Failed to configure virtual ethdev %s\n",
+ name);
+
+ rte_eth_dev_info_get(port->port_id, &port->dev_info);
+ }
+
+ if (test_params.bond_port_id == INVALID_PORT_ID) {
+ retval = rte_eth_bond_create(BONDED_DEV_NAME, 0, rte_socket_id());
+
+ TEST_ASSERT(retval >= 0, "Failed to create bonded ethdev %s",
+ BONDED_DEV_NAME);
+
+ test_params.bond_port_id = retval;
+
+ TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id,
+ &default_pmd_conf, 0), "Failed to configure bonding device\n");
+
+ rte_eth_dev_info_get(test_params.bond_port_id,
+ &test_params.bond_dev_info);
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+check_environment(void)
+{
+ return TEST_SUCCESS;
+}
+
+static int
+test_rssconf_executor(int (*test_func)(void))
+{
+ int test_result;
+
+ /* Check if environment is clean. Fail to launch a test if there was
+ * a critical error before that prevented to reset environment. */
+ TEST_ASSERT_SUCCESS(check_environment(),
+ "Refusing to launch test in dirty environment.");
+
+ RTE_VERIFY(test_func != NULL);
+ test_result = (*test_func)();
+
+ /* If test succeed check if environment wast left in good condition. */
+ if (test_result == TEST_SUCCESS)
+ test_result = check_environment();
+
+ /* Reset environment in case test failed to do that. */
+ if (test_result != TEST_SUCCESS) {
+ TEST_ASSERT_SUCCESS(remove_slaves_and_stop_bonded_device(),
+ "Failed to stop bonded device");
+ }
+
+ return test_result;
+}
+
+static int
+test_setup_wrapper(void)
+{
+ return test_rssconf_executor(&test_setup);
+}
+
+static int
+test_rss_wrapper(void)
+{
+ return test_rssconf_executor(&test_rss);
+}
+
+static int
+test_rss_lazy_wrapper(void)
+{
+ return test_rssconf_executor(&test_rss_lazy);
+}
+
+static struct unit_test_suite link_bonding_rssconf_test_suite = {
+ .suite_name = "RSS Dynamic Configuration for Bonding Unit Test Suite",
+ .setup = test_setup,
+ .unit_test_cases = {
+ TEST_CASE_NAMED("test_setup", test_setup_wrapper),
+ TEST_CASE_NAMED("test_rss", test_rss_wrapper),
+ TEST_CASE_NAMED("test_rss_lazy", test_rss_lazy_wrapper),
+ { NULL, NULL, NULL, NULL, NULL } /**< NULL terminate unit test array */
+ }
+};
+
+static int
+test_link_bonding_rssconf(void)
+{
+ return unit_test_suite_runner(&link_bonding_rssconf_test_suite);
+}
+
+static struct test_command test_link_bonding_rssconf_cmd = {
+ .command = "link_bonding_rssconf_autotest",
+ .callback = test_link_bonding_rssconf,
+};
+
+REGISTER_TEST_COMMAND(test_link_bonding_rssconf_cmd);
--
1.7.9.5
Tomasz Kulasek
2015-06-19 14:13:16 UTC
Permalink
Queue stats mapping is used in example application. This patch adds propagation
of mapping over the slaves, and fills bonding port's stats with a sum of
corresponding values taken from bonded slaves, when stats are requested for
bonding port.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/bonding/rte_eth_bond_pmd.c | 30 +++++++++++++++++++++++++++++-
1 file changed, 29 insertions(+), 1 deletion(-)

diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 5de2f30..90a2ac7 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -1774,7 +1774,7 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
{
struct bond_dev_private *internals = dev->data->dev_private;
struct rte_eth_stats slave_stats;
- int i;
+ int i, j;

for (i = 0; i < internals->slave_count; i++) {
rte_eth_stats_get(internals->slaves[i].port_id, &slave_stats);
@@ -1793,6 +1793,15 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
stats->rx_pause_xon += slave_stats.rx_pause_xon;
stats->tx_pause_xoff += slave_stats.tx_pause_xoff;
stats->rx_pause_xoff += slave_stats.rx_pause_xoff;
+
+ for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) {
+ stats->q_ipackets[j] += slave_stats.q_ipackets[j];
+ stats->q_opackets[j] += slave_stats.q_ipackets[j];
+ stats->q_ibytes[j] += slave_stats.q_ipackets[j];
+ stats->q_obytes[j] += slave_stats.q_ipackets[j];
+ stats->q_errors[j] += slave_stats.q_ipackets[j];
+ }
+
}
}

@@ -2103,6 +2112,24 @@ bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev,
return 0;
}

+static int
+bond_ethdev_queue_stats_mapping_set(struct rte_eth_dev *dev,
+ uint16_t queue_id, uint8_t stat_idx, uint8_t is_rx)
+{
+ int i;
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ for (i = 0; i < internals->slave_count; i++)
+ if (is_rx)
+ rte_eth_dev_set_rx_queue_stats_mapping (
+ internals->slaves[i].port_id, queue_id, stat_idx);
+ else
+ rte_eth_dev_set_tx_queue_stats_mapping (
+ internals->slaves[i].port_id, queue_id, stat_idx);
+
+ return 0;
+}
+
struct eth_dev_ops default_dev_ops = {
.dev_start = bond_ethdev_start,
.dev_stop = bond_ethdev_stop,
@@ -2116,6 +2143,7 @@ struct eth_dev_ops default_dev_ops = {
.link_update = bond_ethdev_link_update,
.stats_get = bond_ethdev_stats_get,
.stats_reset = bond_ethdev_stats_reset,
+ .queue_stats_mapping_set = bond_ethdev_queue_stats_mapping_set,
.promiscuous_enable = bond_ethdev_promiscuous_enable,
.promiscuous_disable = bond_ethdev_promiscuous_disable,
.reta_update = bond_ethdev_rss_reta_update,
--
1.7.9.5
Tomasz Kulasek
2015-06-19 14:13:17 UTC
Permalink
Per queue statistics are already implemented for ring device, but with static
mapping (stat_idx == queue_id).

This fix is required, if you want to use ring device in test application and is
used only to point that per queue statistics are provided for this device.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/ring/rte_eth_ring.c | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index d7e7d9c..975bba7 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -335,6 +335,16 @@ eth_rss_hash_conf_get(struct rte_eth_dev *dev,
return 0;
}

+static int
+eth_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id,
+ __rte_unused uint8_t stat_idx,
+ __rte_unused uint8_t is_rx)
+{
+ /* Do nothing, just return ok */
+ return 0;
+}
+
static const struct eth_dev_ops ops = {
.dev_start = eth_dev_start,
.dev_stop = eth_dev_stop,
@@ -347,6 +357,7 @@ static const struct eth_dev_ops ops = {
.rx_queue_release = eth_queue_release,
.tx_queue_release = eth_queue_release,
.link_update = eth_link_update,
+ .queue_stats_mapping_set = eth_queue_stats_mapping_set,
.stats_get = eth_stats_get,
.stats_reset = eth_stats_reset,
.mac_addr_remove = eth_mac_addr_remove,
--
1.7.9.5
Tomasz Kulasek
2015-06-19 14:13:18 UTC
Permalink
This application allows you to test RSS configuration for bonded devices
changing configuration dynamically, during receiving packets on slaves and
observe the changes in its distribution over queues.

After initialization process, all accessible ports are attached to one bonding
as slaves.

Monitor screen is divided into five main parts:
- Port selection (on the very top)
- RSS Configuration for selected port including hash function, key and RETA
- Incoming packets statistics
- Incoming packets list for selected port
- Status bar with contextual information about selected part

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
examples/bond_rss/Makefile | 59 +++
examples/bond_rss/bondrss.c | 293 ++++++++++++++
examples/bond_rss/bondrss.h | 163 ++++++++
examples/bond_rss/config.c | 251 ++++++++++++
examples/bond_rss/ui.c | 920 +++++++++++++++++++++++++++++++++++++++++++
5 files changed, 1686 insertions(+)
create mode 100644 examples/bond_rss/Makefile
create mode 100644 examples/bond_rss/bondrss.c
create mode 100644 examples/bond_rss/bondrss.h
create mode 100644 examples/bond_rss/config.c
create mode 100644 examples/bond_rss/ui.c

diff --git a/examples/bond_rss/Makefile b/examples/bond_rss/Makefile
new file mode 100644
index 0000000..db457a3
--- /dev/null
+++ b/examples/bond_rss/Makefile
@@ -0,0 +1,59 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# binary name
+APP = bondrss
+
+# all source are stored in SRCS-y
+SRCS-y := bondrss.c
+SRCS-y += ui.c
+SRCS-y += config.c
+
+CFLAGS += $(WERROR_FLAGS)
+
+# workaround for a gcc bug with noreturn attribute
+# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603
+ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y)
+CFLAGS_main.o += -Wno-return-type
+endif
+
+LDLIBS += -lncurses
+
+include $(RTE_SDK)/mk/rte.extapp.mk
diff --git a/examples/bond_rss/bondrss.c b/examples/bond_rss/bondrss.c
new file mode 100644
index 0000000..2dc9979
--- /dev/null
+++ b/examples/bond_rss/bondrss.c
@@ -0,0 +1,293 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "bondrss.h"
+
+#define RSS_HASH_KEY_LENGTH 40
+
+static const struct rte_eth_conf port_conf_default = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_RSS,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled. */
+ .hw_ip_checksum = 0, /**< IP checksum offload disabled. */
+ .hw_vlan_filter = 1, /**< VLAN filtering enabled. */
+ .hw_vlan_strip = 1, /**< VLAN strip enabled. */
+ .hw_vlan_extend = 0, /**< Extended VLAN disabled. */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */
+ .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */
+ .enable_scatter = 0, /**< scatter rx disabled */
+ },
+ .rx_adv_conf = {
+ .rss_conf = {
+ .rss_key = NULL,
+ .rss_hf = ETH_RSS_IPV4,
+ },
+ },
+};
+
+static const struct rte_eth_conf port_conf_bonding = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_RSS,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled. */
+ .hw_ip_checksum = 0, /**< IP checksum offload disabled. */
+ .hw_vlan_filter = 1, /**< VLAN filtering enabled. */
+ .hw_vlan_strip = 1, /**< VLAN strip enabled. */
+ .hw_vlan_extend = 0, /**< Extended VLAN disabled. */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */
+ .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */
+ .enable_scatter = 0, /**< scatter rx disabled */
+ },
+ .rx_adv_conf = {
+ .rss_conf = {
+ .rss_key = NULL,
+ .rss_hf = ETH_RSS_IPV6,
+ },
+ },
+};
+
+/*
+ * Configurable number of RX/TX queues.
+ */
+int nb_rxq = 4; /**< Number of RX queues per port. */
+int nb_txq = 1; /**< Number of TX queues per port. */
+
+int bond_port_id = -1; /**< Bonded device port id */
+
+int nb_ports;
+struct rte_port *ports;
+rte_atomic64_t counter; /**< Received packet's counter */
+
+/*
+ * Initializes a given port using global settings and with the rx buffers
+ * coming from the mbuf_pool passed as parameter
+ */
+static inline
+int init_port(uint8_t port_id, struct rte_mempool *mbuf_pool,
+ struct rte_eth_conf port_conf)
+{
+
+ int retval;
+ uint16_t q;
+ struct rte_eth_rxconf rxconf;
+
+ struct rte_port *port;
+
+ if (port_id >= rte_eth_dev_count())
+ return -1;
+
+ port = &ports[port_id];
+ port->nb_rx_queues = nb_rxq;
+ port->nb_tx_queues = nb_txq;
+
+ memcpy(&(port->dev_conf), &port_conf, sizeof(port_conf));
+ retval = rte_eth_dev_configure(port_id, port->nb_rx_queues,
+ port->nb_tx_queues, &port_conf);
+
+ if (retval != 0)
+ return retval;
+
+ rte_eth_dev_info_get(port_id, &(port->dev_info));
+ rxconf = (port->dev_info).default_rxconf;
+ rxconf.rx_free_thresh = 32;
+
+ for (q = 0; q < port->nb_rx_queues; q++) {
+ retval = rte_eth_rx_queue_setup(port_id, q, RX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id), &rxconf,
+ mbuf_pool);
+ if (retval < 0)
+ return retval;
+ }
+
+ for (q = 0; q < port->nb_tx_queues; q++) {
+ retval = rte_eth_tx_queue_setup(port_id, q, TX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id),
+ &((port->dev_info).default_txconf));
+ if (retval < 0)
+ return retval;
+ }
+
+ port->bond_mode = -1;
+ port->promiscuous = 0;
+ port->enabled = 1;
+
+ return 0;
+}
+
+static int
+rx_loop(__attribute__((unused)) void *dummy)
+{
+ uint8_t port_id;
+ int nq;
+ int n;
+
+ for (;;)
+ for (port_id = 0; port_id < nb_ports; port_id++) {
+ /* Pool only bonding ports */
+ if (ports[port_id].bond_mode >= 0)
+ for (nq = 0; nq < ports[port_id].nb_rx_queues; nq++) {
+ struct rte_mbuf *bufs[BURST_SIZE];
+
+ const uint16_t nb_rx = rte_eth_rx_burst(port_id, nq, bufs,
+ BURST_SIZE);
+ if (unlikely(nb_rx == 0))
+ continue;
+
+ ports[port_id].rxq_ipackets[nq] += nb_rx;
+ for (n = 0; n < nb_rx; n++) {
+ record_pkt(port_id, nq, bufs[n]);
+ rte_pktmbuf_free(bufs[n]);
+ }
+ }
+ }
+
+ return 0;
+}
+
+static uint16_t
+rx_callback(uint8_t port_id, uint16_t qidx, struct rte_mbuf **pkts,
+ uint16_t nb_pkts, uint16_t max_pkts __rte_unused, void *_ __rte_unused)
+{
+ int n;
+
+ ports[port_id].rxq_ipackets[qidx] += nb_pkts;
+ for (n = 0; n < nb_pkts; n++) {
+ rte_atomic64_inc(&counter);
+ pkts[n]->udata64 = counter.cnt;
+ record_pkt(port_id, qidx, pkts[n]);
+ }
+
+ return nb_pkts;
+}
+
+int
+main(int argc, char *argv[])
+{
+ struct rte_mempool *mbuf_pool;
+ unsigned lcore_id = 0;
+ uint8_t port_id, queue_id;
+
+ /* init EAL */
+ int ret = rte_eal_init(argc, argv);
+ if (ret < 0)
+ rte_exit(EXIT_FAILURE, "Error with EAL initialization\n");
+ argc -= ret;
+ argv += ret;
+
+ nb_ports = rte_eth_dev_count();
+ if (nb_ports < 1)
+ rte_exit(EXIT_FAILURE,
+ "At least one port must be available to create a bonding\n");
+
+ mbuf_pool = rte_mempool_create("MBUF_POOL", NUM_MBUFS * nb_ports, MBUF_SIZE,
+ MBUF_CACHE_SIZE,
+ sizeof(struct rte_pktmbuf_pool_private),
+ rte_pktmbuf_pool_init, NULL,
+ rte_pktmbuf_init, NULL, rte_socket_id(), 0);
+ if (mbuf_pool == NULL)
+ rte_exit(EXIT_FAILURE, "Cannot create mbuf pool\n");
+
+ /* Configuration of Ethernet ports. */
+ ports = rte_zmalloc("bondrss: ports",
+ sizeof(struct rte_port) * RTE_MAX_ETHPORTS,
+ RTE_CACHE_LINE_SIZE);
+ if (ports == NULL)
+ rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) failed\n",
+ RTE_MAX_ETHPORTS);
+
+ /* enabled allocated ports */
+ for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++)
+ ports[port_id].enabled = 0;
+
+ /* initialize all ports */
+ for (port_id = 0; port_id < nb_ports; port_id++) {
+ if (init_port(port_id, mbuf_pool, port_conf_default) != 0)
+ rte_exit(EXIT_FAILURE, "Cannot init port %"PRIu8"\n", port_id);
+
+ /* Add rx callbacks to the slave's port */
+ for (queue_id = 0; queue_id < ports[port_id].nb_rx_queues; queue_id++)
+ rte_eth_add_rx_callback(port_id, queue_id, rx_callback, NULL);
+
+ }
+
+ /* create bonding port */
+ bond_port_id = rte_eth_bond_create("eth_bond", 0, 0);
+ if (bond_port_id < 0)
+ rte_exit(EXIT_FAILURE, "Cannot create bonding port\n");
+
+ for (port_id = 0; port_id < nb_ports; port_id++)
+ rte_eth_bond_slave_add(bond_port_id, port_id);
+
+ /* count again */
+ nb_ports = rte_eth_dev_count();
+
+ init_port(bond_port_id, mbuf_pool, port_conf_bonding);
+
+ /* start bonding port*/
+ ret = rte_eth_dev_start(bond_port_id);
+
+ /* enable promiscuous by default */
+ rte_eth_promiscuous_enable(bond_port_id);
+
+ for (port_id = 0; port_id < nb_ports; port_id++) {
+ ports[port_id].bond_mode = rte_eth_bond_mode_get(port_id);
+ ports[port_id].promiscuous = rte_eth_promiscuous_get(port_id);
+
+ /* map queues to stats */
+ if (ports[port_id].bond_mode >= 0) {
+ for (queue_id = 0; queue_id < ports[port_id].nb_rx_queues;
+ queue_id++) {
+ rte_eth_dev_set_rx_queue_stats_mapping(port_id, queue_id,
+ queue_id);
+ }
+ }
+
+ }
+
+ /* Initialize received packet's counter */
+ rte_atomic64_init(&counter);
+
+ if (rte_lcore_count() <= 1)
+ rte_exit(EXIT_FAILURE, "More than one free lcore is needed\n");
+
+ lcore_id = rte_get_next_lcore(rte_lcore_id(), 1, 0);
+ rte_eal_remote_launch(rx_loop, NULL, lcore_id);
+
+ /* call lcore_main on master core only */
+ ui_loop();
+
+ return 0;
+}
diff --git a/examples/bond_rss/bondrss.h b/examples/bond_rss/bondrss.h
new file mode 100644
index 0000000..305a690
--- /dev/null
+++ b/examples/bond_rss/bondrss.h
@@ -0,0 +1,163 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef BONDRSS_H_
+#define BONDRSS_H_
+
+#include <stdint.h>
+#include <inttypes.h>
+#include <rte_eal.h>
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_lcore.h>
+#include <rte_mbuf.h>
+#include <rte_malloc.h>
+
+#include <rte_eth_bond.h>
+
+#define RX_RING_SIZE 128
+#define TX_RING_SIZE 512
+
+#define NUM_MBUFS 8191
+#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)
+#define MBUF_CACHE_SIZE 250
+#define BURST_SIZE 32
+
+/*
+ * Configurable number of RX/TX queues.
+ */
+extern int nb_rxq; /**< Number of RX queues per port. */
+extern int nb_txq; /**< Number of TX queues per port. */
+
+extern int bond_port_id; /**< Bonded device port id */
+extern int bond_mode;
+
+struct rss_type_info {
+ char str[32];
+ uint64_t rss_type;
+};
+
+static struct rss_type_info rss_type_table[] = {
+ {"ipv4", ETH_RSS_IPV4},
+ {"ipv4-frag", ETH_RSS_FRAG_IPV4},
+ {"ipv4-tcp", ETH_RSS_NONFRAG_IPV4_TCP},
+ {"ipv4-udp", ETH_RSS_NONFRAG_IPV4_UDP},
+ {"ipv4-sctp", ETH_RSS_NONFRAG_IPV4_SCTP},
+ {"ipv4-other", ETH_RSS_NONFRAG_IPV4_OTHER},
+ {"ipv6", ETH_RSS_IPV6},
+ {"ipv6-frag", ETH_RSS_FRAG_IPV6},
+ {"ipv6-tcp", ETH_RSS_NONFRAG_IPV6_TCP},
+ {"ipv6-udp", ETH_RSS_NONFRAG_IPV6_UDP},
+ {"ipv6-sctp", ETH_RSS_NONFRAG_IPV6_SCTP},
+ {"ipv6-other", ETH_RSS_NONFRAG_IPV6_OTHER},
+ {"l2-payload", ETH_RSS_L2_PAYLOAD},
+ {"ipv6-ex", ETH_RSS_IPV6_EX},
+ {"ipv6-tcp-ex", ETH_RSS_IPV6_TCP_EX},
+ {"ipv6-udp-ex", ETH_RSS_IPV6_UDP_EX},
+};
+
+struct rss_type_fn {
+ uint8_t enabled;
+ struct rss_type_info *info;
+};
+
+/**
+ * Buffer for last received packets
+ */
+#define PKT_RECORD_SIZE 256
+
+struct pkt_info {
+ uint64_t n; /**< Packet number */
+ uint8_t port_id;
+ uint8_t queue_id;
+ uint64_t ol_flags; /**< Offload features. */
+ uint32_t rss; /**< RSS hash result if RSS enabled */
+};
+
+struct pkt_record {
+ uint64_t count; /**< Total number of received packets */
+ int top;
+ struct pkt_info pkt_info[PKT_RECORD_SIZE];
+};
+
+/**
+ * The data structure associated with each port.
+ */
+struct rte_port {
+ struct rte_eth_dev_info dev_info; /**< PCI info + driver name */
+ struct rte_eth_conf dev_conf; /**< Port configuration. */
+
+ uint16_t nb_rx_queues; /**< Total number of rx queues */
+ uint16_t nb_tx_queues; /**< Total number of tx queues*/
+
+ int bond_mode;
+ struct ether_addr addr;
+
+ int rss_type_count;
+ struct rss_type_fn rss_type_fn[RTE_DIM(rss_type_table)];
+ struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE];
+ uint8_t rss_key[52];
+
+
+ struct rte_eth_stats stats;
+ uint64_t rxq_ipackets[128];
+
+ struct pkt_record record;
+
+ uint8_t rss; /**< RSS enabled for port */
+ uint8_t promiscuous; /**< promiscuous mode enabled for port */
+ uint8_t enabled; /**< port is enabled */
+};
+
+extern int nb_ports;
+extern struct rte_port *ports;
+extern rte_atomic64_t counter; /**< Received packet's counter */
+
+void update_port_info(uint8_t port_id);
+
+int rss_enable(uint8_t port_id, int enabled);
+
+int key_set(uint8_t port_id, uint8_t *key, uint8_t len);
+int key_set_random(uint8_t port_id, uint8_t len);
+
+int reta_set_default(uint8_t port_id);
+int reta_set_random(uint8_t port_id);
+int reta_set_all(uint8_t port_id, uint8_t value);
+int reta_set_weights(uint8_t port_id, uint64_t *weights);
+
+struct pkt_info *record_pkt(uint8_t port_id, int queue_id, struct rte_mbuf *mb);
+
+void ui_loop(void);
+
+
+#endif /* BONDRSS_H_ */
diff --git a/examples/bond_rss/config.c b/examples/bond_rss/config.c
new file mode 100644
index 0000000..39c00e5
--- /dev/null
+++ b/examples/bond_rss/config.c
@@ -0,0 +1,251 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <time.h>
+#include <stdlib.h>
+#include "bondrss.h"
+
+static const struct rte_eth_conf port_conf_default = {
+ .rxmode = {
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled. */
+ .hw_ip_checksum = 0, /**< IP checksum offload disabled. */
+ .hw_vlan_filter = 1, /**< VLAN filtering enabled. */
+ .hw_vlan_strip = 1, /**< VLAN strip enabled. */
+ .hw_vlan_extend = 0, /**< Extended VLAN disabled. */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */
+ .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */
+ .enable_scatter = 0, /**< scatter rx disabled */
+ },
+ .rx_adv_conf = {
+ .rss_conf = {
+ .rss_key = NULL,
+ .rss_hf = ETH_RSS_IPV4,
+ },
+ },
+};
+
+
+void
+update_port_info(uint8_t port_id)
+{
+ int i;
+ struct rte_port *port;
+
+ port = &ports[port_id];
+
+ rte_eth_dev_info_get(port_id, &(port->dev_info));
+
+ port->promiscuous = rte_eth_promiscuous_get(port_id);
+
+ /* Update device information */
+ rte_eth_dev_info_get(port_id, &port->dev_info);
+ rte_eth_macaddr_get(port_id, &port->addr);
+
+ port->dev_conf.rx_adv_conf.rss_conf.rss_key = port->rss_key;
+ port->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 40;
+ rte_eth_dev_rss_hash_conf_get(port_id,
+ &(port->dev_conf.rx_adv_conf.rss_conf));
+
+ /* select all fields to be fetched */
+ for (i = 0; i < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; i++)
+ port->reta_conf[i].mask = ~0LL;
+
+ rte_eth_dev_rss_reta_query(port_id, port->reta_conf,
+ port->dev_info.reta_size);
+
+}
+
+/**
+ * Try to enable RSS for port_id.
+ */
+int
+rss_enable(uint8_t port_id, int enabled)
+{
+ struct rte_port *port;
+ int retval;
+
+ port = &ports[port_id];
+
+ rte_eth_dev_stop(port_id);
+
+ if (enabled) {
+ port->dev_conf.rxmode.mq_mode = ETH_MQ_RX_RSS;
+ port->dev_conf.rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IPV4;
+ } else {
+ port->dev_conf.rxmode.mq_mode = 0;
+ port->dev_conf.rx_adv_conf.rss_conf.rss_hf = 0;
+ }
+
+ retval = rte_eth_dev_configure(port_id, port->nb_rx_queues,
+ port->nb_tx_queues, &port->dev_conf);
+ if (retval != 0)
+ return retval;
+
+ retval = rte_eth_dev_start(port_id);
+
+ return 0;
+}
+
+int
+key_set(uint8_t port_id, uint8_t *key, uint8_t len)
+{
+ struct rte_eth_rss_conf rss_conf;
+
+ int i;
+
+ rss_conf.rss_key = NULL;
+ rss_conf.rss_key_len = 0;
+ i = rte_eth_dev_rss_hash_conf_get(port_id, &rss_conf);
+
+ rss_conf.rss_key = key;
+ rss_conf.rss_key_len = len;
+
+ i = rte_eth_dev_rss_hash_update(port_id,
+ &rss_conf);
+
+ return i;
+}
+
+/**
+ * Set random RSS key value
+ */
+int
+key_set_random(uint8_t port_id, uint8_t len)
+{
+ int i;
+ uint8_t key[40];
+
+ for (i = 0; i < len; i++)
+ key[i] = rand() % 256;
+
+ return key_set(port_id, key, len);
+}
+
+/**
+ * Set random RETA values
+ */
+int
+reta_set_random(uint8_t port_id)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ int i, j;
+
+ int reta_size = ports[port_id].dev_info.reta_size;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) {
+ /* select all fields to set */
+ reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ reta_conf[i].reta[j] = rand() % ports[port_id].nb_rx_queues;
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
+
+/**
+ * Set default RETA values
+ */
+int
+reta_set_default(uint8_t port_id)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ int i, j;
+
+ int reta_size = ports[port_id].dev_info.reta_size;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) {
+ /* select all fields to set */
+ reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ reta_conf[i].reta[j] = j % nb_rxq;
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
+
+/**
+ * Fill the RETA with values
+ */
+int
+reta_set_all(uint8_t port_id, uint8_t value)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ int i, j;
+
+ int reta_size = ports[port_id].dev_info.reta_size;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) {
+ /* select all fields to set */
+ reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ reta_conf[i].reta[j] = value;
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
+
+/**
+ * Set the RETA basing on weights table
+ */
+int
+reta_set_weights(uint8_t port_id, uint64_t *weights)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ unsigned i, j, k;
+
+ unsigned reta_size = ports[port_id].dev_info.reta_size;
+
+ uint64_t sum = 0, sum2;
+
+ for (i = 0; i < ports[port_id].nb_rx_queues; i++)
+ sum += weights[i];
+ sum2 = sum * (ports[port_id].nb_rx_queues-1);
+
+ if (sum2 == 0)
+ return 0;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++)
+ reta_conf[i].mask = ~0LL;
+ k = 0;
+ for (i = 0; i < ports[port_id].nb_rx_queues; i++) {
+ for (j = 0; k < reta_size &&
+ j < ((sum - weights[i]) * reta_size + 1) / sum2; j++) {
+ reta_conf[k/RTE_RETA_GROUP_SIZE].reta[k%RTE_RETA_GROUP_SIZE] = i;
+ k++;
+ }
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
diff --git a/examples/bond_rss/ui.c b/examples/bond_rss/ui.c
new file mode 100644
index 0000000..90cae09
--- /dev/null
+++ b/examples/bond_rss/ui.c
@@ -0,0 +1,920 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_atomic.h>
+
+#include <ncurses.h>
+#include <string.h>
+#include <signal.h>
+#include <sys/ioctl.h>
+#include <menu.h>
+#include "bondrss.h"
+
+WINDOW *win_footer = NULL;
+WINDOW *win_header = NULL;
+WINDOW *win_list = NULL;
+WINDOW *win_stats = NULL;
+WINDOW *win_reta = NULL;
+WINDOW *win_err = NULL;
+
+int scr_lines = 0;
+int scr_cols = 0;
+
+unsigned win_header_lines;
+unsigned win_list_lines;
+unsigned win_stats_lines;
+unsigned win_reta_lines;
+unsigned win_err_lines = 0;
+
+const char *ui_status_text;
+static char *ui_err_text[5];
+
+struct rte_port *selected_port;
+static uint8_t selected_port_id;
+
+struct rte_port *focused_port;
+static uint8_t focused_port_id;
+
+#define UI_MODE_PORT 0
+#define UI_MODE_RSS_FN 1
+#define UI_MODE_RSS_KEY 2
+#define UI_MODE_RETA 3
+#define UI_MODE_HELP 4
+
+int ui_mode = UI_MODE_PORT;
+
+static uint16_t _is_lock;
+static uint16_t _is_ready;
+
+static uint16_t _do_configure;
+static uint16_t _do_repaint;
+static uint16_t _do_repaint_stats;
+static uint16_t _do_repaint_list;
+static uint16_t _do_repaint_info;
+static uint16_t _do_repaint_err;
+
+
+struct help_page {
+ const char *title;
+ const char *body;
+};
+
+static struct help_page help_pages[3] = {
+ {
+ .title = "About",
+
+ .body =
+ "\n\n"
+ " ______ ______ ______ _____ ______ ______ ______\n"
+ "| | | \\ / | | \\ | | \\ \\ | | \\ \\ | | | \\ / | / |\n"
+ "| |--| < | | | | | | | | | | | | | |__| | '------. '------.\n"
+ "|_|__|_/ \\_|__|_/ |_| |_| |_|_/_/ |_| \\_\\ ____|_/ ____|_/\n"
+ "\n\n"
+ "This application allows you to test RSS configuration for "
+ "bonded devices\nchanging configuration dynamically, during "
+ "receiving packets on slaves and\nobserve the changes in its "
+ "distribution over queues.\n"
+ "\n"
+
+ },
+ {
+ .title = "Introduction",
+ .body = "\n\n"
+ "After initialization process, all accessible ports are "
+ "attached to one\nbonding as slaves.\n"
+ "\n"
+
+ "Monitor screen is divided into five main parts:\n"
+ " - Port selection (on the very top)\n"
+ " - RSS Configuration for selected port including hash "
+ "function, key and\n RETA\n"
+ " - Incoming packets statistics\n"
+ " - Incoming packets list for selected port\n"
+ " - Status bar with contextual information about selected "
+ "part\n"
+ "\n\n"
+
+ "[up], [down] arrows key lets you highlight a part "
+ "and parameter to change.\n\n"
+ },
+ {
+ .title = "Key bindings",
+ .body =
+ "_Port_selection_\n\n"
+ " [left], [right] - select port,\n"
+ " [space] - force turning on/off RSS mode,\n"
+ " [p] - turn on/off promiscuous mode,\n"
+ " [c] - clear statistics and incoming packet list,\n"
+ "\n"
+ "_RSS_hash_function_\n\n"
+ " [1]-[9] - toggle selected hash function,\n"
+ " [a] - select all,\n"
+ "\n"
+ "_RSS_key_\n\n"
+ " [1]-[4] - set one of predefined key value,\n"
+ " [r] - randomize value,\n"
+ "\n"
+ "_RSS_RETA_\n\n"
+ " [number] - fill RETA with number,\n"
+ " [r] - randomize value,\n"
+ " [d] - set default value,"
+
+ }
+};
+
+static int
+ui_lock(int wait)
+{
+ int result;
+
+ while (!(result = rte_atomic16_cmpset(&_is_lock, 0, 1)) && wait)
+ rte_pause();
+
+ return result;
+}
+
+static int
+ui_unlock(void)
+{
+ return rte_atomic16_cmpset(&_is_lock, 1, 0);
+}
+
+/**
+ * Predefined key values
+ */
+static uint8_t RSS_KEY[4][40] = {
+ { 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D,
+ 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+ 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B,
+ 0xBE, 0xAC, 0x01, 0xFA
+ },
+ { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00,
+ },
+ { 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A,
+ 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A,
+ 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A,
+ 0x6D, 0x5A, 0x6D, 0x5A,
+ },
+ { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C,
+ 0x0D, 0x0E, 0x0F, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18,
+ 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F, 0x20, 0x21, 0x22, 0x23, 0x24,
+ 0x25, 0x26, 0x27, 0x28,
+ }
+};
+
+static void
+win_err_addline(const char *text)
+{
+ int i;
+
+ if (win_err_lines < 5) {
+ win_err_lines++;
+ _do_configure = 1;
+ } else {
+ free(ui_err_text[0]);
+ for (i = 1; i < 5; i++)
+ ui_err_text[i - 1] = ui_err_text[i];
+ }
+
+ ui_err_text[win_err_lines - 1] = strdup(text);
+ _do_repaint_err = 1;
+}
+
+static void
+win_err_fini(void)
+{
+ int i;
+ for (i = 0; i < (int)win_err_lines; i++)
+ free(ui_err_text[i]);
+}
+
+/**
+ * Draw full information for port
+ */
+static void
+win_header_draw(uint8_t port_id)
+{
+ int i, j;
+ attr_t a;
+
+ update_port_info(selected_port_id);
+
+ /* Draw title bar */
+ if (ui_mode == UI_MODE_PORT)
+ wattron(win_header, COLOR_PAIR(2));
+ else
+ wattron(win_header, COLOR_PAIR(4));
+
+ mvwprintw(win_header, 0, 0, "::");
+
+ for (i = 0; i < nb_ports; i++) {
+ if (ports[i].enabled) {
+ const char *name = (ports[i].bond_mode >= 0 ? "BOND" : "SLAVE");
+
+ if (i == port_id) {
+ wattron(win_header, A_REVERSE);
+ wprintw(win_header, " %s-%d ", name, (unsigned) i);
+ wattroff(win_header, A_REVERSE);
+ } else
+ wprintw(win_header, " %s-%d ", name, (unsigned) i);
+ }
+ }
+
+ for (i = scr_cols - getcurx(win_header) - 3; i > 0; i--)
+ waddch(win_header, ':');
+
+ waddch(win_header,
+ (ports[port_id].rss ? 'R' : '-'));
+ waddch(win_header, ports[port_id].bond_mode >= 0 ?
+ ports[port_id].bond_mode + '0' : '-');
+ waddch(win_header, ports[port_id].promiscuous == 0 ? '-' : 'P');
+
+ if (ui_mode == UI_MODE_PORT)
+ wattroff(win_header, COLOR_PAIR(2));
+ else
+ wattroff(win_header, COLOR_PAIR(4));
+
+ /* Redraw RSS-Fn */
+ selected_port->rss_type_count = 0;
+ for (i = 0; i < (int) RTE_DIM(rss_type_table); i++) {
+ if (selected_port->dev_info.flow_type_rss_offloads
+ & rss_type_table[i].rss_type) {
+ selected_port->rss_type_fn[selected_port->rss_type_count].info =
+ &rss_type_table[i];
+ selected_port->rss_type_fn[selected_port->rss_type_count].enabled =
+ ((selected_port->dev_conf.rx_adv_conf.rss_conf.rss_hf
+ & rss_type_table[i].rss_type) ? 1 : 0);
+ selected_port->rss_type_count++;
+ }
+ }
+
+ a = (ui_mode == UI_MODE_RSS_FN ? COLOR_PAIR(2) : COLOR_PAIR(1));
+
+ wattron(win_header, a);
+ mvwprintw(win_header, 1, 0, "FN: ");
+ for (i = 0; i < selected_port->rss_type_count; i++) {
+ if (selected_port->rss_type_fn[i].enabled)
+ wattron(win_header, COLOR_PAIR(3));
+ waddstr(win_header, selected_port->rss_type_fn[i].info->str);
+ if (selected_port->rss_type_fn[i].enabled)
+ wattron(win_header, a);
+ waddch(win_header, ' ');
+ }
+ wattroff(win_header, a);
+
+ /* Redraw RSS-Key */
+ if (ui_mode == UI_MODE_RSS_KEY)
+ wattron(win_header, COLOR_PAIR(2));
+ mvwprintw(win_header, 2, 0, "KEY: ");
+ for (i = 0; i < 40; i++)
+ wprintw(win_header, "%02X",
+ selected_port->dev_conf.rx_adv_conf.rss_conf.rss_key[i]);
+ if (ui_mode == UI_MODE_RSS_KEY)
+ wattroff(win_header, COLOR_PAIR(2));
+
+
+ /* Redraw RETA window */
+ int idx, shift;
+
+ if (ui_mode == UI_MODE_RETA)
+ wattron(win_reta, COLOR_PAIR(2));
+
+ for (j = 0; j < selected_port->dev_info.reta_size / 16; j++) {
+ wmove(win_reta, j, 0);
+ for (i = 0; i < 16; i++) {
+ idx = (j*16 + i) / RTE_RETA_GROUP_SIZE;
+ shift = (j*16 + i) % RTE_RETA_GROUP_SIZE;
+ waddch(win_reta, ACS_VLINE);
+ wprintw(win_reta, "%d", selected_port->reta_conf[idx].reta[shift]);
+ }
+ waddch(win_reta, ACS_VLINE);
+ }
+
+ if (ui_mode == UI_MODE_RETA)
+ wattroff(win_reta, COLOR_PAIR(2));
+ wnoutrefresh(win_reta);
+
+
+ /* Stats repaint */
+ if (_do_repaint_stats) {
+ uint64_t total;
+
+ rte_eth_stats_get(selected_port_id, &(selected_port->stats));
+
+ wmove(win_stats, 0, 0);
+ total = 0;
+ for (i = 0; i < selected_port->nb_rx_queues; i++) {
+ wprintw(win_stats, "Queue %d: %10"PRIu64
+ " (%10" PRIu64 ")\n", i, selected_port->rxq_ipackets[i],
+ selected_port->stats.q_ipackets[i]);
+ total += selected_port->rxq_ipackets[i];
+ }
+
+ wprintw(win_stats, " Total: %10"PRIu64
+ " (%10" PRIu64 ")\n", total, selected_port->stats.ipackets);
+
+ _do_repaint_stats = 0;
+ wnoutrefresh(win_stats);
+ }
+
+ if (_do_repaint_err && win_err_lines > 0) {
+ for (i = 0; i < (int)win_err_lines; i++) {
+ mvwprintw(win_err, i, 0, ui_err_text[i]);
+ wclrtoeol(win_err);
+ }
+ _do_repaint_err = 0;
+ wnoutrefresh(win_err);
+ }
+
+ mvwhline(win_header, win_header_lines - 1, 0, 0, scr_cols);
+}
+
+static void
+win_footer_draw(const char *text)
+{
+ if (text != NULL)
+ ui_status_text = text;
+
+ wclear(win_footer);
+ wmove(win_footer, 0, 0);
+ wprintw(win_footer, ui_status_text);
+
+ wprintw(win_footer, "; [arrows]navigate; [q]quit");
+ wnoutrefresh(win_footer);
+}
+
+static void
+win_list_draw(void)
+{
+ unsigned n, i;
+
+ struct pkt_info *pkt_info;
+
+ struct rte_port *port = &ports[focused_port_id];
+ struct pkt_record *record = &(port->record);
+
+ n = (win_list_lines > record->count) ? record->count : win_list_lines;
+
+ wmove(win_list, 0, 0);
+
+ for (i = 0; i < n; i++) {
+
+ pkt_info = &record->pkt_info[(record->top - n + i + PKT_RECORD_SIZE)
+ % PKT_RECORD_SIZE];
+
+ wmove(win_list, i, 0);
+
+ wprintw(win_list, "%5"PRIu64, (unsigned) pkt_info->n);
+ waddch(win_list, ACS_VLINE);
+
+ wprintw(win_list, "%2d: ", (unsigned) pkt_info->queue_id);
+ wprintw(win_list, "%08x ", (unsigned) pkt_info->rss);
+
+ if (pkt_info->ol_flags != 0) {
+ unsigned rxf;
+ const char *name;
+
+ for (rxf = 0; rxf < sizeof(pkt_info->ol_flags) * 8; rxf++) {
+ if ((pkt_info->ol_flags & (1ULL << rxf)) == 0)
+ continue;
+ name = rte_get_rx_ol_flag_name(1ULL << rxf);
+ if (name == NULL)
+ continue;
+ wprintw(win_list, "%s ", name);
+ }
+ }
+ wclrtoeol(win_list);
+ }
+
+ wclrtobot(win_list);
+ wnoutrefresh(win_list);
+}
+
+static void
+ui_repaint(void)
+{
+ switch (ui_mode) {
+ default:
+ win_header_draw(selected_port_id);
+ wnoutrefresh(win_header);
+
+ if (_do_repaint_list) {
+ win_list_draw();
+ _do_repaint_list = 0;
+ }
+
+ break;
+ }
+
+ win_footer_draw(NULL);
+ _do_repaint = 0;
+}
+
+static void
+ui_mode_set(int mode)
+{
+ ui_mode = (mode + 4) % 4;
+ switch (ui_mode) {
+ case UI_MODE_PORT:
+ ui_status_text = "Port: [p]promiscuous; [c]clear stats";
+ break;
+ case UI_MODE_RSS_FN:
+ ui_status_text = "RSS Fn: [1-9]toggle fn";
+ break;
+ case UI_MODE_RETA:
+ ui_status_text =
+ "RETA: [d]set default; [r]randomize; [0-9]fill table with value";
+ break;
+ case UI_MODE_RSS_KEY:
+ ui_status_text = "RSS Key: [r]randomize; [1-4]select predefined key";
+ break;
+ }
+ _do_repaint = 1;
+}
+
+static void
+ui_configure(void)
+{
+ int win_reta_cols = (16 * 2 + 1);
+ win_reta_lines = selected_port->dev_info.reta_size / 16;
+ win_stats_lines = selected_port->nb_rx_queues + 1;
+ win_header_lines = win_reta_lines;
+ if (win_header_lines < win_stats_lines)
+ win_header_lines = win_stats_lines;
+ win_header_lines += 5;
+ win_list_lines = scr_lines - win_header_lines - win_err_lines - 1;
+
+ if (win_footer == NULL) {
+ win_footer = newwin(1, scr_cols, scr_lines - 1, 0);
+ ui_status_text = "";
+ wbkgdset(win_footer, COLOR_PAIR(1));
+ } else {
+ wresize(win_footer, 1, scr_cols);
+ mvwin(win_footer, scr_lines - 1, 0);
+ }
+
+ if (win_err == NULL) {
+ win_err = newwin(1, scr_cols, scr_lines - 1 - win_err_lines, 0);
+ ui_status_text = "";
+ wbkgdset(win_err, COLOR_PAIR(5));
+ } else {
+ wresize(win_err, win_err_lines, scr_cols);
+ mvwin(win_err, scr_lines - win_err_lines - 1, 0);
+ }
+
+
+ if (win_header == NULL) {
+ win_header = newwin(win_header_lines, scr_cols, 0, 0);
+ wbkgdset(win_header, COLOR_PAIR(1));
+ } else
+ wresize(win_header, win_header_lines, scr_cols);
+
+ if (win_stats == NULL)
+ win_stats = subwin(win_header, win_stats_lines,
+ scr_cols - win_reta_cols - 7, 4, win_reta_cols + 7);
+ else
+ wresize(win_stats, win_stats_lines, scr_cols - win_reta_cols - 7);
+
+ if (win_reta == NULL)
+ win_reta = subwin(win_header, win_reta_lines, win_reta_cols,
+ 4, 5);
+ else
+ wresize(win_reta, win_reta_lines, win_reta_cols);
+
+ if (win_list == NULL)
+ win_list = newwin(win_list_lines, scr_cols, win_header_lines, 0);
+ else {
+ wresize(win_list, win_list_lines, scr_cols);
+ mvwin(win_list, win_header_lines, 0);
+ }
+
+ wclear(win_header);
+ wclear(win_reta);
+
+ _do_configure = 0;
+ _do_repaint = 1;
+ _do_repaint_list = 1;
+ _do_repaint_stats = 1;
+ _do_repaint_info = 1;
+ _do_repaint_err = 1;
+
+}
+
+static void
+ui_resize(void)
+{
+ if (COLS != scr_cols || LINES != scr_lines) {
+ scr_cols = COLS;
+ scr_lines = LINES;
+ _do_configure = 1;
+ }
+}
+
+static void
+ui_update(void)
+{
+ if (ui_lock(1)) {
+ if (_is_ready) {
+ if (_do_configure)
+ ui_configure();
+ if (_do_repaint) {
+ ui_repaint();
+ doupdate();
+ }
+ }
+ ui_unlock();
+ }
+}
+
+static void
+ui_fini(void)
+{
+ win_err_fini();
+ endwin();
+}
+
+static void
+ui_init(void)
+{
+ initscr();
+ start_color();
+ init_pair(1, COLOR_YELLOW, COLOR_BLUE);
+ init_pair(2, COLOR_BLACK, COLOR_CYAN);
+ init_pair(3, COLOR_BLACK, COLOR_YELLOW);
+ init_pair(4, COLOR_BLACK, COLOR_WHITE);
+ init_pair(5, COLOR_YELLOW, COLOR_RED);
+
+ cbreak();
+ noecho();
+ curs_set(FALSE);
+ keypad(stdscr, TRUE);
+ timeout(100);
+
+ scr_cols = COLS;
+ scr_lines = LINES;
+ _do_configure = 1;
+ _is_ready = 1;
+}
+
+static int
+port_select(uint8_t port_id)
+{
+ if (ports[port_id].enabled == 1) {
+ focused_port_id = selected_port_id = port_id;
+
+ selected_port = &ports[selected_port_id];
+ focused_port = &ports[focused_port_id];
+
+ update_port_info(selected_port_id);
+
+ _do_configure = 1;
+ _do_repaint = 1;
+ _do_repaint_info = 1;
+ _do_repaint_list = 1;
+ _do_repaint_stats = 1;
+ return selected_port_id;
+ }
+ return -1;
+}
+
+/**
+ * Record packet from mbuf
+ */
+struct pkt_info *
+record_pkt(uint8_t port_id, int queue_id, struct rte_mbuf *mb)
+{
+ struct rte_port *port = &ports[port_id];
+ struct pkt_record *record = &(port->record);
+ struct pkt_info *pkt_info = &(record->pkt_info[record->top]);
+
+ ui_lock(1);
+
+ record->count++;
+ pkt_info->n = mb->udata64;
+ pkt_info->ol_flags = mb->ol_flags;
+ pkt_info->queue_id = queue_id;
+ pkt_info->rss = mb->hash.rss;
+
+ record->top = (record->top + 1) % PKT_RECORD_SIZE;
+
+ _do_repaint = 1;
+ _do_repaint_list = 1;
+ _do_repaint_stats = 1;
+ ui_unlock();
+
+ return pkt_info;
+}
+
+static void
+ui_help(void) {
+ timeout(-1);
+
+ WINDOW *win_help_border = NULL;
+ WINDOW *win_help = NULL;
+ char title[256];
+ int page = 0;
+ int page_prev;
+ int page_next;
+ int c;
+ int cols;
+ int lines;
+ int top;
+ int left;
+
+ endwin();
+ initscr();
+ ui_resize();
+ ui_update();
+
+ cols = scr_cols > 80 ? 80 : scr_cols;
+ lines = scr_lines > 25 ? 25 : scr_lines;
+ top = (scr_lines - lines) / 2;
+ left = (scr_cols - cols) / 2;
+
+ win_help_border = newwin(lines, cols, top, left);
+ win_help = derwin(win_help_border, lines - 2, cols - 4, 1, 2);
+ wbkgdset(win_help_border, COLOR_PAIR(2));
+ wbkgdset(win_help, COLOR_PAIR(2));
+
+ do {
+ page_prev = (page + RTE_DIM(help_pages) - 1) % RTE_DIM(help_pages);
+ page_next = (page + 1) % RTE_DIM(help_pages);
+
+ wclear(win_help_border);
+ box(win_help_border, 0 , 0);
+
+ sprintf(title, "[ Help: %s ]", help_pages[page].title);
+ mvwprintw(win_help_border, 0, (cols - strlen(title)) / 2 - 2, "%s",
+ title);
+
+ sprintf(title, "< %s ]", help_pages[page_prev].title);
+ mvwprintw(win_help_border, lines - 1, 1, "%s", title);
+
+ sprintf(title, "[ Page %d of %d ]", page + 1, (int)RTE_DIM(help_pages));
+ mvwprintw(win_help_border, lines - 1, (cols - strlen(title)) / 2 - 2,
+ "%s", title);
+
+ sprintf(title, "[ %s >", help_pages[page_next].title);
+ mvwprintw(win_help_border, lines - 1, cols - strlen(title) - 1, "%s",
+ title);
+
+ wrefresh(win_help_border);
+
+ wclear(win_help);
+
+ wmove(win_help, 0, 0);
+
+ wprintw(win_help,
+ help_pages[page].body
+ );
+
+ wrefresh(win_help);
+
+ switch (c = getch()) {
+
+ case KEY_RESIZE:
+ endwin();
+ initscr();
+ ui_resize();
+ ui_update();
+
+ cols = scr_cols > 80 ? 80 : scr_cols;
+ lines = scr_lines > 25 ? 25 : scr_cols;
+ top = (scr_lines - lines) / 2;
+ left = (scr_cols - cols) / 2;
+
+ wresize(win_help_border, lines, cols);
+ wresize(win_help, lines - 2, cols - 4);
+ break;
+
+ case KEY_LEFT:
+ page = page_prev;
+ break;
+
+ case KEY_RIGHT:
+ page = page_next;
+ break;
+
+ default:
+ c = -1; /* Exit */
+ break;
+ }
+
+
+ } while (c != -1);
+
+ timeout(100);
+ delwin(win_help);
+ delwin(win_help_border);
+ _do_configure = 1;
+ _do_repaint = 1;
+}
+
+/* main processing loop */
+void
+ui_loop(void)
+{
+ int c;
+ int i;
+ int port_id;
+ int refresh = 0;
+
+ ui_init();
+ port_select(bond_port_id);
+
+ ui_mode_set(UI_MODE_PORT);
+
+ while ((c = getch()) != 'q') {
+ switch (c) {
+ case -1:
+ refresh++;
+ if (refresh % 10) {
+ _do_configure = 1;
+ }
+ ui_update();
+ break;
+
+ case 'c':
+ /* clear stats */
+ ui_lock(1);
+
+ rte_atomic64_clear(&counter);
+ for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) {
+ if (ports[port_id].enabled) {
+ ports[port_id].record.count = 0;
+ ports[port_id].record.top = 0;
+ for (i = 0; i < ports[port_id].nb_rx_queues; i++)
+ ports[port_id].rxq_ipackets[i] = 0;
+ rte_eth_stats_reset(port_id);
+ }
+ }
+
+ _do_repaint = 1;
+ ui_unlock();
+ break;
+
+ case ' ':
+ focused_port->rss ^= 1;
+ rss_enable(focused_port_id, focused_port->rss);
+ _do_configure = 1;
+ _do_repaint = 1;
+ break;
+
+ case 'p':
+ if (focused_port->promiscuous)
+ rte_eth_promiscuous_disable(focused_port_id);
+ else
+ rte_eth_promiscuous_enable(focused_port_id);
+ focused_port->promiscuous ^= 1;
+ _do_configure = 1;
+ _do_repaint = 1;
+ break;
+
+ case KEY_RESIZE:
+ ui_resize();
+ break;
+
+ case KEY_UP:
+ ui_mode_set(ui_mode - 1);
+ break;
+ case KEY_DOWN:
+ ui_mode_set(ui_mode + 1);
+ break;
+ case 'h':
+ case '?':
+ ui_help();
+ break;
+
+ default:
+ switch (ui_mode) {
+ case UI_MODE_PORT:
+ switch (c) {
+ case KEY_LEFT:
+ for (i = 1; i < nb_ports; i++)
+ if (port_select(
+ (selected_port_id - i + nb_ports) % nb_ports)
+ != -1)
+ break;
+ break;
+
+ case KEY_RIGHT:
+ for (i = 1; i < nb_ports; i++)
+ if (port_select(
+ (selected_port_id + i + nb_ports) % nb_ports)
+ != -1)
+ break;
+ break;
+
+ default:
+ i = (c - '1');
+ port_select(i);
+ break;
+ }
+ break;
+
+ case UI_MODE_RSS_KEY:
+ switch (c) {
+ case 'r':
+ i = key_set_random(focused_port_id, 40);
+ if (i < 0)
+ win_err_addline("Cannot update RSS key");
+ break;
+
+ default:
+ c = c - '1';
+ if (c >= 0 && c < (int) RTE_DIM(RSS_KEY)) {
+ i = key_set(focused_port_id, RSS_KEY[c], 40);
+ if (i < 0)
+ win_err_addline("Cannot update RSS key");
+ }
+ break;
+ }
+ update_port_info(focused_port_id);
+ break;
+
+ case UI_MODE_RETA:
+ switch (c) {
+ case 'r':
+ reta_set_random(focused_port_id);
+ break;
+ case 'd':
+ reta_set_default(focused_port_id);
+ break;
+ default:
+ c = c - '0';
+ reta_set_all(focused_port_id, c);
+ break;
+ }
+ break;
+
+ case UI_MODE_RSS_FN:
+ switch (c) {
+ case 'a':
+ /* Set all */
+ focused_port->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ ports[focused_port_id].dev_info.flow_type_rss_offloads;
+ rte_eth_dev_rss_hash_update(focused_port_id,
+ &focused_port->dev_conf.rx_adv_conf.rss_conf);
+ break;
+
+ default:
+ c -= '1';
+ if (c >= 0 && c < focused_port->rss_type_count) {
+ uint64_t rss_type = 0;
+
+ focused_port->rss_type_fn[c].enabled ^= 1; /* toggle */
+
+ for (i = 0; i < focused_port->rss_type_count; i++)
+ if (focused_port->rss_type_fn[i].enabled)
+ rss_type |=
+ focused_port->rss_type_fn[i].info->rss_type;
+ focused_port->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ rss_type;
+
+ rte_eth_dev_rss_hash_update(focused_port_id,
+ &focused_port->dev_conf.rx_adv_conf.rss_conf);
+
+ }
+ }
+ break;
+
+ }
+ _do_repaint = 1;
+ break;
+ }
+ }
+ ui_fini();
+}
--
1.7.9.5
Tomasz Kulasek
2015-06-19 14:13:19 UTC
Permalink
Signed-off-by: Tomasz Kulasek <***@intel.com>
---
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
index 96e554f..03baf90 100644
--- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
+++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
@@ -188,7 +188,7 @@ conditions are not met. If a user wishes to monitor individual slaves then they
must register callbacks with that slave directly.

The link bonding library also supports devices which do not implement link
-status change interrupts, this is achieve by polling the devices link status at
+status change interrupts, this is achieved by polling the devices link status at
a defined period which is set using the ``rte_eth_bond_link_monitoring_set``
API, the default polling interval is 10ms. When a device is added as a slave to
a bonding device it is determined using the ``RTE_PCI_DRV_INTR_LSC`` flag
@@ -286,7 +286,7 @@ and UDP protocols for load balancing.
Using Link Bonding Devices
--------------------------

-The librte_pmd_bond library support two modes of device creation, the libraries
+The librte_pmd_bond library supports two modes of device creation, the libraries
export full C API or using the EAL command line to statically configure link
bonding devices at application startup. Using the EAL option it is possible to
use link bonding functionality transparently without specific knowledge of the
@@ -299,7 +299,7 @@ Using the Poll Mode Driver from an Application

Using the librte_pmd_bond libraries API it is possible to dynamically create
and manage link bonding device from within any application. Link bonding
-device are created using the ``rte_eth_bond_create`` API which requires a
+devices are created using the ``rte_eth_bond_create`` API which requires a
unique device name, the link bonding mode to initial the device in and finally
the socket Id which to allocate the devices resources onto. After successful
creation of a bonding device it must be configured using the generic Ethernet
@@ -362,7 +362,7 @@ The different options are:
mode=2

* slave: Defines the PMD device which will be added as slave to the bonded
- device. This option can be selected multiple time, for each device to be
+ device. This option can be selected multiple times, for each device to be
added as a slave. Physical devices should be specified using their PCI
address, in the format domain:bus:devid.function
--
1.7.9.5
Tomasz Kulasek
2015-06-19 14:13:20 UTC
Permalink
Documentation update about implementation details and requirements for Dynamic
RSS Configuration for Bonding.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 34 ++++++++++++++++++--
1 file changed, 32 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
index 03baf90..46f0296 100644
--- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
+++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
@@ -1,5 +1,5 @@
.. BSD LICENSE
- Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+ Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
All rights reserved.

Redistribution and use in source and binary forms, with or without
@@ -173,7 +173,28 @@ After a slave device is added to a bonded device slave is stopped using
``rte_eth_dev_stop`` and then reconfigured using ``rte_eth_dev_configure``
the RX and TX queues are also reconfigured using ``rte_eth_tx_queue_setup`` /
``rte_eth_rx_queue_setup`` with the parameters use to configure the bonding
-device.
+device. If RSS is enabled for bonding device, this mode is also enabled on new
+slave and configured as well.
+
+Setting up multi-queue mode for bonding device to RSS, makes it fully
+RSS-capable, so all slaves are synchronized with its configuration. This mode is
+intended to provide RSS configuration on slaves transparent for client
+application implementation.
+
+Bonding device stores its own version of RSS settings i.e. RETA, RSS hash
+function and RSS key, used to set up its slaves. That let to define the meaning
+of RSS configuration of bonding device as desired configuration of whole bonding
+(as one unit), without pointing any of slave inside. It is required to ensure
+consistency and made it more errorproof.
+
+RSS hash function set for bonding device, is a maximal set of RSS hash functions
+supported by all bonded slaves. RETA size is a GCD of all its RETA's sizes, so
+it can be easily used as a pattern providing expected behavior, even if slave
+RETAs' sizes are different. If RSS Key is not set for bonded device, it's not
+changed on the slaves and default key for device is used.
+
+All settings are managed through the bonding port API and always are propagated
+in one direction (from bonding to slaves).

Link Status Change Interrupts / Polling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -207,6 +228,15 @@ these parameters.
A bonding device must have a minimum of one slave before the bonding device
itself can be started.

+To use a bonding device dynamic RSS configuration feature effectively, it is
+also required, that all slaves should be RSS-capable and support, at least one
+common hash function available for each of them. Changing RSS key is only
+possible, when all slave devices support the same key size.
+
+To prevent inconsistency on how slaves process packets, once a device is added
+to a bonding device, RSS configuration should be managed through the bonding
+device API, and not directly on the slave.
+
Like all other PMD, all functions exported by a PMD are lock-free functions
that are assumed not to be invoked in parallel on different logical cores to
work on the same target object.
--
1.7.9.5
Doherty, Declan
2015-06-26 07:33:00 UTC
Permalink
-----Original Message-----
Sent: Friday, June 19, 2015 7:13 AM
Subject: [dpdk-dev] [PATCH v2 0/8] Dynamic RSS Configuration for Bonding
OVERVIEW
--------
1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding
device fully RSS-capable, so all slaves are synchronized with its configuration.
This mode is intended to provide RSS configuration as known from "dynamic RSS
configuration for one port" and made slaves transparent for client application
implementation.
2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not
synchronized. That provides an ability to configure them manually. This mode may
be useful when application wants to manage RSS in an unusual way and the
consistency of RSS configuration for slaves isn't required.
Turning on/off RSS mode for slaves when bonding is started is not possible.
Other RSS configuration is propagated over slaves, when bonding device API is
used to do it.
- added support for keys other than 40 bytes long,
- now, if RSS key is not set for bonding, it is not set also for slaves,
- fix - full initial RSS configuration before any slave is added was not
possible due to the initially zeroed flow_type_rss_offloads for bonding,
- fix - changed error to warning when slave is synchronizing due to the
bonding's initial configuration (to allow use slaves' drivers not supporting
dynamic RSS configuration in bonding),
- some code cleanups,
- updated documentation,
bond: rss dynamic configuration
ring: dynamic rss configuration
test: dynamic rss configuration
bond: queue stats mapping
ring: queue stats mapping set dummy implementation
examples: dynamic rss configuration for bonding
doc: fixed spellings and typos
doc: dynamic rss configuration for bonding
app/test/Makefile | 1 +
app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 42 +-
drivers/net/bonding/rte_eth_bond_api.c | 27 +
drivers/net/bonding/rte_eth_bond_pmd.c | 239 ++++-
drivers/net/bonding/rte_eth_bond_private.h | 12 +
drivers/net/ring/rte_eth_ring.c | 133 ++-
examples/bond_rss/Makefile | 59 ++
examples/bond_rss/bondrss.c | 293 +++++++
examples/bond_rss/bondrss.h | 163 ++++
examples/bond_rss/config.c | 251 ++++++
examples/bond_rss/ui.c | 920 ++++++++++++++++++++
12 files changed, 2789 insertions(+), 25 deletions(-)
create mode 100644 app/test/test_link_bonding_rssconf.c
create mode 100644 examples/bond_rss/Makefile
create mode 100644 examples/bond_rss/bondrss.c
create mode 100644 examples/bond_rss/bondrss.h
create mode 100644 examples/bond_rss/config.c
create mode 100644 examples/bond_rss/ui.c
--
1.7.9.5
Ack
Thomas Monjalon
2015-06-28 21:54:24 UTC
Permalink
Please use checkpatch to fix formatting.
Tomasz Kulasek
2015-06-29 14:50:36 UTC
Permalink
OVERVIEW
--------
1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding
device fully RSS-capable, so all slaves are synchronized with its configuration.
This mode is intended to provide RSS configuration as known from "dynamic RSS
configuration for one port" and made slaves transparent for client application
implementation.

2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not
synchronized. That provides an ability to configure them manually. This mode may
be useful when application wants to manage RSS in an unusual way and the
consistency of RSS configuration for slaves isn't required.

Turning on/off RSS mode for slaves when bonding is started is not possible.
Other RSS configuration is propagated over slaves, when bonding device API is
used to do it.

v3 changes:
- checkpatch cleanups

v2 changes:
- added support for keys other than 40 bytes long,
- now, if RSS key is not set for bonding, it is not set also for slaves,
- fix - full initial RSS configuration before any slave is added was not
possible due to the initially zeroed flow_type_rss_offloads for bonding,
- fix - changed error to warning when slave is synchronizing due to the
bonding's initial configuration (to allow use slaves' drivers not supporting
dynamic RSS configuration in bonding),
- some code cleanups,
- updated documentation,

Tomasz Kulasek (8):
bonding: rss dynamic configuration
ring: dynamic rss configuration
test: dynamic rss configuration
bonding: queue stats mapping
ring: queue stats mapping set dummy implementation
examples: dynamic rss configuration for bonding
doc: fixed spellings and typos
doc: dynamic rss configuration for bonding

app/test/Makefile | 1 +
app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 42 +-
drivers/net/bonding/rte_eth_bond_api.c | 28 +
drivers/net/bonding/rte_eth_bond_pmd.c | 235 ++++-
drivers/net/bonding/rte_eth_bond_private.h | 12 +
drivers/net/ring/rte_eth_ring.c | 130 ++-
examples/bond_rss/Makefile | 59 ++
examples/bond_rss/bondrss.c | 293 ++++++
examples/bond_rss/bondrss.h | 163 ++++
examples/bond_rss/config.c | 251 ++++++
examples/bond_rss/ui.c | 945 ++++++++++++++++++++
12 files changed, 2808 insertions(+), 25 deletions(-)
create mode 100644 app/test/test_link_bonding_rssconf.c
create mode 100644 examples/bond_rss/Makefile
create mode 100644 examples/bond_rss/bondrss.c
create mode 100644 examples/bond_rss/bondrss.h
create mode 100644 examples/bond_rss/config.c
create mode 100644 examples/bond_rss/ui.c
--
1.7.9.5
Tomasz Kulasek
2015-06-29 14:50:37 UTC
Permalink
Bonding device implements independent management of RSS settings. It
stores its own copies of settings i.e. RETA, RSS hash function and RSS
key. It’s required to ensure consistency.

1) RSS hash function set for bonding device is maximal set of RSS hash
functions supported by all bonded devices. That mean, to have RSS support
for bonding, all slaves should be RSS-capable.

2) RSS key is propagated over the slaves "as is".

3) RETA for bonding is an internal table managed by bonding API, and is
used as a pattern to set up slaves. Its size is GCD of all RETA sizes, so
it can be easily used as a pattern providing expected behavior, even if
slaves RETA sizes are different.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/bonding/rte_eth_bond_api.c | 28 ++++
drivers/net/bonding/rte_eth_bond_pmd.c | 205 ++++++++++++++++++++++++++--
drivers/net/bonding/rte_eth_bond_private.h | 12 ++
3 files changed, 231 insertions(+), 14 deletions(-)

diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c
index d810ec4..22eb575 100644
--- a/drivers/net/bonding/rte_eth_bond_api.c
+++ b/drivers/net/bonding/rte_eth_bond_api.c
@@ -303,6 +303,9 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id)
internals->rx_offload_capa = 0;
internals->tx_offload_capa = 0;

+ /* Initially allow to choose any offload type */
+ internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK;
+
memset(internals->active_slaves, 0, sizeof(internals->active_slaves));
memset(internals->slaves, 0, sizeof(internals->slaves));

@@ -366,6 +369,11 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)

rte_eth_dev_info_get(slave_port_id, &dev_info);

+ /* We need to store slaves reta_size to be able to synchronize RETA for all
+ * slave devices even if its sizes are different.
+ */
+ internals->slaves[internals->slave_count].reta_size = dev_info.reta_size;
+
if (internals->slave_count < 1) {
/* if MAC is not user defined then use MAC of first slave add to
* bonded device */
@@ -379,9 +387,16 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
/* Make primary slave */
internals->primary_port = slave_port_id;

+ /* Inherit queues settings from first slave */
+ internals->nb_rx_queues = slave_eth_dev->data->nb_rx_queues;
+ internals->nb_tx_queues = slave_eth_dev->data->nb_tx_queues;
+
+ internals->reta_size = dev_info.reta_size;
+
/* Take the first dev's offload capabilities */
internals->rx_offload_capa = dev_info.rx_offload_capa;
internals->tx_offload_capa = dev_info.tx_offload_capa;
+ internals->flow_type_rss_offloads = dev_info.flow_type_rss_offloads;

} else {
/* Check slave link properties are supported if props are set,
@@ -400,8 +415,19 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
}
internals->rx_offload_capa &= dev_info.rx_offload_capa;
internals->tx_offload_capa &= dev_info.tx_offload_capa;
+ internals->flow_type_rss_offloads &= dev_info.flow_type_rss_offloads;
+
+ /* RETA size is GCD of all slaves RETA sizes, so, if all sizes will be
+ * the power of 2, the lower one is GCD
+ */
+ if (internals->reta_size > dev_info.reta_size)
+ internals->reta_size = dev_info.reta_size;
+
}

+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf &=
+ internals->flow_type_rss_offloads;
+
internals->slave_count++;

/* Update all slave devices MACs*/
@@ -528,6 +554,8 @@ __eth_bond_slave_remove_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
if (internals->slave_count == 0) {
internals->rx_offload_capa = 0;
internals->tx_offload_capa = 0;
+ internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK;
+ internals->reta_size = 0;
}
return 0;
}
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 989e878..cd23f42 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -1310,6 +1310,23 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev,
if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC)
slave_eth_dev->data->dev_conf.intr_conf.lsc = 1;

+ /* If RSS is enabled for bonding, try to enable it for slaves */
+ if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) {
+ if (bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len
+ != 0) {
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len =
+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len;
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key =
+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key;
+ } else {
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL;
+ }
+
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ slave_eth_dev->data->dev_conf.rxmode.mq_mode |= ETH_MQ_RX_RSS;
+ }
+
/* Configure device */
errval = rte_eth_dev_configure(slave_eth_dev->data->port_id,
bonded_eth_dev->data->nb_rx_queues,
@@ -1361,6 +1378,30 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev,
return -1;
}

+ /* If RSS is enabled for bonding, synchronize RETA */
+ if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) {
+ int i;
+ struct bond_dev_private *internals;
+
+ internals = bonded_eth_dev->data->dev_private;
+
+ for (i = 0; i < internals->slave_count; i++) {
+ if (internals->slaves[i].port_id == slave_eth_dev->data->port_id) {
+ errval = rte_eth_dev_rss_reta_update(
+ slave_eth_dev->data->port_id,
+ &internals->reta_conf[0],
+ internals->slaves[i].reta_size);
+ if (errval != 0) {
+ RTE_LOG(WARNING, PMD,
+ "rte_eth_dev_rss_reta_update on slave port %d fails (err %d)."
+ " RSS Configuration for bonding may be inconsistent.\n",
+ slave_eth_dev->data->port_id, errval);
+ }
+ break;
+ }
+ }
+ }
+
/* If lsc interrupt is set, check initial slave's link status */
if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC)
bond_ethdev_lsc_event_callback(slave_eth_dev->data->port_id,
@@ -1578,6 +1619,9 @@ bond_ethdev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)

dev_info->rx_offload_capa = internals->rx_offload_capa;
dev_info->tx_offload_capa = internals->tx_offload_capa;
+ dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads;
+
+ dev_info->reta_size = internals->reta_size;
}

static int
@@ -1959,21 +2003,132 @@ bond_ethdev_lsc_event_callback(uint8_t port_id, enum rte_eth_event_type type,
}
}

+static int
+bond_ethdev_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ unsigned i, j;
+ int result = 0;
+ int slave_reta_size;
+ unsigned reta_count;
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ if (reta_size != internals->reta_size)
+ return -EINVAL;
+
+ /* Copy RETA table */
+ reta_count = reta_size / RTE_RETA_GROUP_SIZE;
+
+ for (i = 0; i < reta_count; i++) {
+ internals->reta_conf[i].mask = reta_conf[i].mask;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ internals->reta_conf[i].reta[j] = reta_conf[i].reta[j];
+ }
+
+ /* Fill rest of array */
+ for (; i < RTE_DIM(internals->reta_conf); i += reta_count)
+ memcpy(&internals->reta_conf[i], &internals->reta_conf[0],
+ sizeof(internals->reta_conf[0]) * reta_count);
+
+ /* Propagate RETA over slaves */
+ for (i = 0; i < internals->slave_count; i++) {
+ slave_reta_size = internals->slaves[i].reta_size;
+ result = rte_eth_dev_rss_reta_update(internals->slaves[i].port_id,
+ &internals->reta_conf[0], slave_reta_size);
+ if (result < 0)
+ return result;
+ }
+
+ return 0;
+}
+
+static int
+bond_ethdev_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ int i, j;
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ if (reta_size != internals->reta_size)
+ return -EINVAL;
+
+ /* Copy RETA table */
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++)
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ reta_conf[i].reta[j] = internals->reta_conf[i].reta[j];
+
+ return 0;
+}
+
+static int
+bond_ethdev_rss_hash_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ int i, result = 0;
+ struct bond_dev_private *internals = dev->data->dev_private;
+ struct rte_eth_rss_conf bond_rss_conf;
+
+ memcpy(&bond_rss_conf, rss_conf, sizeof(struct rte_eth_rss_conf));
+
+ bond_rss_conf.rss_hf &= internals->flow_type_rss_offloads;
+
+ if (bond_rss_conf.rss_hf != 0)
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = bond_rss_conf.rss_hf;
+
+ if (bond_rss_conf.rss_key && bond_rss_conf.rss_key_len <
+ sizeof(internals->rss_key)) {
+ if (bond_rss_conf.rss_key_len == 0)
+ bond_rss_conf.rss_key_len = 40;
+ internals->rss_key_len = bond_rss_conf.rss_key_len;
+ memcpy(internals->rss_key, bond_rss_conf.rss_key,
+ internals->rss_key_len);
+ }
+
+ for (i = 0; i < internals->slave_count; i++) {
+ result = rte_eth_dev_rss_hash_update(internals->slaves[i].port_id,
+ &bond_rss_conf);
+ if (result < 0)
+ return result;
+ }
+
+ return 0;
+}
+
+static int
+bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ rss_conf->rss_key_len = internals->rss_key_len;
+ if (rss_conf->rss_key)
+ memcpy(rss_conf->rss_key, internals->rss_key, internals->rss_key_len);
+
+ return 0;
+}
+
struct eth_dev_ops default_dev_ops = {
- .dev_start = bond_ethdev_start,
- .dev_stop = bond_ethdev_stop,
- .dev_close = bond_ethdev_close,
- .dev_configure = bond_ethdev_configure,
- .dev_infos_get = bond_ethdev_info,
- .rx_queue_setup = bond_ethdev_rx_queue_setup,
- .tx_queue_setup = bond_ethdev_tx_queue_setup,
- .rx_queue_release = bond_ethdev_rx_queue_release,
- .tx_queue_release = bond_ethdev_tx_queue_release,
- .link_update = bond_ethdev_link_update,
- .stats_get = bond_ethdev_stats_get,
- .stats_reset = bond_ethdev_stats_reset,
- .promiscuous_enable = bond_ethdev_promiscuous_enable,
- .promiscuous_disable = bond_ethdev_promiscuous_disable
+ .dev_start = bond_ethdev_start,
+ .dev_stop = bond_ethdev_stop,
+ .dev_close = bond_ethdev_close,
+ .dev_configure = bond_ethdev_configure,
+ .dev_infos_get = bond_ethdev_info,
+ .rx_queue_setup = bond_ethdev_rx_queue_setup,
+ .tx_queue_setup = bond_ethdev_tx_queue_setup,
+ .rx_queue_release = bond_ethdev_rx_queue_release,
+ .tx_queue_release = bond_ethdev_tx_queue_release,
+ .link_update = bond_ethdev_link_update,
+ .stats_get = bond_ethdev_stats_get,
+ .stats_reset = bond_ethdev_stats_reset,
+ .promiscuous_enable = bond_ethdev_promiscuous_enable,
+ .promiscuous_disable = bond_ethdev_promiscuous_disable,
+ .reta_update = bond_ethdev_rss_reta_update,
+ .reta_query = bond_ethdev_rss_reta_query,
+ .rss_hash_update = bond_ethdev_rss_hash_update,
+ .rss_hash_conf_get = bond_ethdev_rss_hash_conf_get
};

static int
@@ -2054,6 +2209,28 @@ bond_ethdev_configure(struct rte_eth_dev *dev)
int arg_count;
uint8_t port_id = dev - rte_eth_devices;

+ static const uint8_t default_rss_key[40] = {
+ 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D,
+ 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+ 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B,
+ 0xBE, 0xAC, 0x01, 0xFA
+ };
+
+ unsigned i, j;
+
+ /* If RSS is enabled, fill table and key with default values */
+ if (dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) {
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = internals->rss_key;
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 0;
+ memcpy(internals->rss_key, default_rss_key, 40);
+
+ for (i = 0; i < RTE_DIM(internals->reta_conf); i++) {
+ internals->reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ internals->reta_conf[i].reta[j] = j % dev->data->nb_rx_queues;
+ }
+ }
+
/*
* if no kvlist, it means that this bonded device has been created
* through the bonding api.
diff --git a/drivers/net/bonding/rte_eth_bond_private.h b/drivers/net/bonding/rte_eth_bond_private.h
index 45e5c65..98bb64d 100644
--- a/drivers/net/bonding/rte_eth_bond_private.h
+++ b/drivers/net/bonding/rte_eth_bond_private.h
@@ -103,6 +103,8 @@ struct bond_slave_details {
uint8_t last_link_status;
/**< Port Id of slave eth_dev */
struct ether_addr persisted_mac_addr;
+
+ uint16_t reta_size;
};


@@ -155,6 +157,16 @@ struct bond_dev_private {
uint32_t rx_offload_capa; /** Rx offload capability */
uint32_t tx_offload_capa; /** Tx offload capability */

+ /** Bit mask of RSS offloads, the bit offset also means flow type */
+ uint64_t flow_type_rss_offloads;
+
+ uint16_t reta_size;
+ struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_512 /
+ RTE_RETA_GROUP_SIZE];
+
+ uint8_t rss_key[52]; /**< 52-byte hash key buffer. */
+ uint8_t rss_key_len; /**< hash key length in bytes. */
+
struct rte_kvargs *kvlist;
uint8_t slave_update_idx;
};
--
1.7.9.5
Tomasz Kulasek
2015-06-29 14:50:38 UTC
Permalink
This implementation allows to set and read RSS configuration for ring
device, and is used to validate right values propagation over the slaves,
in test units for dynamic RSS configuration for bonding.

It have no impact on packet processing by ring device.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/ring/rte_eth_ring.c | 120 +++++++++++++++++++++++++++++++++++++--
1 file changed, 116 insertions(+), 4 deletions(-)

diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index ada025f..29d7bd8 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -39,6 +39,7 @@
#include <rte_string_fns.h>
#include <rte_dev.h>
#include <rte_kvargs.h>
+#include <rte_spinlock.h>

#define ETH_RING_NUMA_NODE_ACTION_ARG "nodeaction"
#define ETH_RING_ACTION_CREATE "CREATE"
@@ -66,6 +67,17 @@ struct pmd_internals {
struct ring_queue tx_ring_queues[RTE_PMD_RING_MAX_TX_RINGS];

struct ether_addr address;
+
+ /** Bit mask of RSS offloads, the bit offset also means flow type */
+ uint64_t flow_type_rss_offloads;
+
+ rte_spinlock_t rss_lock;
+
+ uint16_t reta_size;
+ struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_128 /
+ RTE_RETA_GROUP_SIZE];
+
+ uint8_t rss_key[40]; /**< 40-byte hash key. */
};


@@ -173,6 +185,8 @@ eth_dev_info(struct rte_eth_dev *dev,
dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues;
dev_info->min_rx_bufsize = 0;
dev_info->pci_dev = NULL;
+ dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads;
+ dev_info->reta_size = internals->reta_size;
}

static void
@@ -234,6 +248,91 @@ static int
eth_link_update(struct rte_eth_dev *dev __rte_unused,
int wait_to_complete __rte_unused) { return 0; }

+static int
+eth_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ int i, j;
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ if (reta_size != internal->reta_size)
+ return -EINVAL;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ /* Copy RETA table */
+ for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) {
+ internal->reta_conf[i].mask = reta_conf[i].mask;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ internal->reta_conf[i].reta[j] = reta_conf[i].reta[j];
+ }
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ int i, j;
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ if (reta_size != internal->reta_size)
+ return -EINVAL;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ /* Copy RETA table */
+ for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) {
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ reta_conf[i].reta[j] = internal->reta_conf[i].reta[j];
+ }
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_hash_update(struct rte_eth_dev *dev, struct rte_eth_rss_conf *rss_conf)
+{
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0)
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ rss_conf->rss_hf & internal->flow_type_rss_offloads;
+
+ if (rss_conf->rss_key)
+ memcpy(internal->rss_key, rss_conf->rss_key, 40);
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ if (rss_conf->rss_key)
+ memcpy(rss_conf->rss_key, internal->rss_key, 40);
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
static const struct eth_dev_ops ops = {
.dev_start = eth_dev_start,
.dev_stop = eth_dev_stop,
@@ -250,6 +349,10 @@ static const struct eth_dev_ops ops = {
.stats_reset = eth_stats_reset,
.mac_addr_remove = eth_mac_addr_remove,
.mac_addr_add = eth_mac_addr_add,
+ .reta_update = eth_rss_reta_update,
+ .reta_query = eth_rss_reta_query,
+ .rss_hash_update = eth_rss_hash_update,
+ .rss_hash_conf_get = eth_rss_hash_conf_get
};

int
@@ -268,6 +371,13 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[],

unsigned i;

+ static const uint8_t default_rss_key[40] = {
+ 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D,
+ 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+ 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B,
+ 0xBE, 0xAC, 0x01, 0xFA
+ };
+
/* do some parameter checking */
if (rx_queues == NULL && nb_rx_queues > 0)
goto error;
@@ -316,12 +426,14 @@ rte_eth_from_rings(const char *name, struct rte_ring *const rx_queues[],

internals->nb_rx_queues = nb_rx_queues;
internals->nb_tx_queues = nb_tx_queues;
- for (i = 0; i < nb_rx_queues; i++) {
+ for (i = 0; i < nb_rx_queues; i++)
internals->rx_ring_queues[i].rng = rx_queues[i];
- }
- for (i = 0; i < nb_tx_queues; i++) {
+ for (i = 0; i < nb_tx_queues; i++)
internals->tx_ring_queues[i].rng = tx_queues[i];
- }
+ internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK;
+ internals->reta_size = RTE_DIM(internals->reta_conf) * RTE_RETA_GROUP_SIZE;
+
+ memcpy(internals->rss_key, default_rss_key, 40);

eth_drv->pci_drv.name = ring_ethdev_driver_name;
eth_drv->pci_drv.id_table = id_table;
--
1.7.9.5
Thomas Monjalon
2015-07-13 12:36:15 UTC
Permalink
Post by Tomasz Kulasek
This implementation allows to set and read RSS configuration for ring
device, and is used to validate right values propagation over the slaves,
in test units for dynamic RSS configuration for bonding.
It have no impact on packet processing by ring device.
Adding some fake RSS to the ring PMD (in order to test bonding) is weird.
The ring PMD is not a driver for testing. Maybe that the null PMD would be
more appropriate.
By the way the current RSS implementation is really bound to Intel devices.
Before applying it to more drivers, we have to make sure it is generic
enough. Maybe the RETA needs more abstraction.
Kulasek, TomaszX
2015-07-13 14:43:25 UTC
Permalink
-----Original Message-----
Sent: Monday, July 13, 2015 14:36
To: Kulasek, TomaszX
Subject: Re: [dpdk-dev] [PATCH v3 2/8] ring: dynamic rss configuration
Post by Tomasz Kulasek
This implementation allows to set and read RSS configuration for ring
device, and is used to validate right values propagation over the
slaves, in test units for dynamic RSS configuration for bonding.
It have no impact on packet processing by ring device.
Adding some fake RSS to the ring PMD (in order to test bonding) is weird.
The ring PMD is not a driver for testing. Maybe that the null PMD would be
more appropriate.
By the way the current RSS implementation is really bound to Intel devices.
Before applying it to more drivers, we have to make sure it is generic
enough. Maybe the RETA needs more abstraction.
This is not RSS implementation, but implementation of Dynamic RSS Configuration, already existing, and well defined in official documentation for single port.
I don't know where you see bounding to Intel device. It's an implementation of official API.

Anyway I will appreciate for any comment and opinion on that and clarification what means more RETA abstraction in context of official API.

I will check the possibility of moving it to the null pmd driver and will prepare new version for easier reviewing. Meantime I'm waiting for more opinions.
Thomas Monjalon
2015-07-13 15:11:09 UTC
Permalink
Post by Kulasek, TomaszX
Post by Thomas Monjalon
Post by Tomasz Kulasek
This implementation allows to set and read RSS configuration for ring
device, and is used to validate right values propagation over the
slaves, in test units for dynamic RSS configuration for bonding.
It have no impact on packet processing by ring device.
Adding some fake RSS to the ring PMD (in order to test bonding) is weird.
The ring PMD is not a driver for testing. Maybe that the null PMD would be
more appropriate.
By the way the current RSS implementation is really bound to Intel devices.
Before applying it to more drivers, we have to make sure it is generic
enough. Maybe the RETA needs more abstraction.
This is not RSS implementation, but implementation of Dynamic RSS
Configuration, already existing, and well defined in official documentation
for single port.
I don't know where you see bounding to Intel device.
It's an implementation of official API.
What do you mean by "official"?
My concern is that the RSS API in ethdev comes from a time where DPDK was
Intel DPDK. I may be wrong but I think that other devices could need
something more generic and better defined.
Post by Kulasek, TomaszX
Anyway I will appreciate for any comment and opinion on that and
clarification what means more RETA abstraction in context of official API.
I will check the possibility of moving it to the null pmd driver and will
prepare new version for easier reviewing. Meantime I'm waiting for more
opinions.
Exact, we need more opinions on this topic.
Kulasek, TomaszX
2015-07-16 13:02:52 UTC
Permalink
-----Original Message-----
Sent: Monday, July 13, 2015 17:11
To: Kulasek, TomaszX
Subject: Re: [dpdk-dev] [PATCH v3 2/8] ring: dynamic rss configuration
Post by Kulasek, TomaszX
Post by Thomas Monjalon
Post by Tomasz Kulasek
This implementation allows to set and read RSS configuration for
ring device, and is used to validate right values propagation over
the slaves, in test units for dynamic RSS configuration for bonding.
It have no impact on packet processing by ring device.
Adding some fake RSS to the ring PMD (in order to test bonding) is
weird.
Post by Kulasek, TomaszX
Post by Thomas Monjalon
The ring PMD is not a driver for testing. Maybe that the null PMD
would be more appropriate.
By the way the current RSS implementation is really bound to Intel devices.
Before applying it to more drivers, we have to make sure it is
generic enough. Maybe the RETA needs more abstraction.
This is not RSS implementation, but implementation of Dynamic RSS
Configuration, already existing, and well defined in official
documentation for single port.
I don't know where you see bounding to Intel device.
It's an implementation of official API.
What do you mean by "official"?
My concern is that the RSS API in ethdev comes from a time where DPDK was
Intel DPDK. I may be wrong but I think that other devices could need
something more generic and better defined.
Post by Kulasek, TomaszX
Anyway I will appreciate for any comment and opinion on that and
clarification what means more RETA abstraction in context of official
API.
Post by Kulasek, TomaszX
I will check the possibility of moving it to the null pmd driver and
will prepare new version for easier reviewing. Meantime I'm waiting
for more opinions.
Exact, we need more opinions on this topic.
Hi Thomas,
I have fixed the error with wrong copy/paste. Also I changed usage of ring pmd with null pmd like you suggested. You are right, it is better place. Thanks.

I sent a v4 patch-set.

This implementation is using an existing ethdev API for RSS and introduces generic enhancements for configuration of bonding slaves.
If other devices need a more generic ethdev API then this should be treated separately and they can submit separate patches for this in a future release.
Tomasz Kulasek
2015-06-29 14:50:39 UTC
Permalink
This test module uses ring device to check right RSS configuration
propagation.

1) Propagation test
a) Set RSS hash function for bonding, fetch RSS hash function from
bonded slave, check if same. Do it for all slaves.
b) Repeat above for RSS key and RETA.

2)Dynamic adding slave to the bonding
a) Remove slave from bonding.
b) Change its configuration to other than bonding.
c) Add slave to the started bonding device.
d) Check if slaves configuration changed.

3) Repeat point 1) and 2) with RSS multi-queue mode enabled/disabled for
bonding port.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
app/test/Makefile | 1 +
app/test/test_link_bonding_rssconf.c | 674 ++++++++++++++++++++++++++++++++++
2 files changed, 675 insertions(+)
create mode 100644 app/test/test_link_bonding_rssconf.c

diff --git a/app/test/Makefile b/app/test/Makefile
index 2e2758c..15926fe 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -136,6 +136,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_ACL) += test_acl.c
ifeq ($(CONFIG_RTE_LIBRTE_PMD_RING),y)
SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_mode4.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_rssconf.c
endif

SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring.c
diff --git a/app/test/test_link_bonding_rssconf.c b/app/test/test_link_bonding_rssconf.c
new file mode 100644
index 0000000..96bc5c6
--- /dev/null
+++ b/app/test/test_link_bonding_rssconf.c
@@ -0,0 +1,674 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <errno.h>
+#include <rte_cycles.h>
+#include <sys/queue.h>
+
+#include <rte_byteorder.h>
+#include <rte_common.h>
+#include <rte_debug.h>
+#include <rte_ethdev.h>
+#include <rte_log.h>
+#include <rte_lcore.h>
+#include <rte_memory.h>
+
+#include <rte_string_fns.h>
+
+#include <rte_eth_ring.h>
+#include <rte_errno.h>
+#include <rte_eth_bond.h>
+
+#include "test.h"
+
+#define SLAVE_COUNT (4)
+
+#define RXTX_RING_SIZE 1024
+#define RXTX_QUEUE_COUNT 4
+
+#define BONDED_DEV_NAME ("rssconf_bond_dev")
+
+#define SLAVE_DEV_NAME_FMT ("rssconf_slave%d")
+#define SLAVE_RXTX_QUEUE_FMT ("rssconf_slave%d_q%d")
+
+#define NUM_MBUFS 8191
+#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)
+#define MBUF_CACHE_SIZE 250
+#define BURST_SIZE 32
+
+#define INVALID_SOCKET_ID (-1)
+#define INVALID_PORT_ID (0xFF)
+#define INVALID_BONDING_MODE (-1)
+
+struct slave_conf {
+ uint8_t port_id;
+ struct rte_eth_dev_info dev_info;
+
+ struct rte_eth_rss_conf rss_conf;
+ uint8_t rss_key[40];
+ struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE];
+
+ uint8_t is_slave;
+ struct rte_ring *rxtx_queue[RXTX_QUEUE_COUNT];
+};
+
+struct link_bonding_rssconf_unittest_params {
+ uint8_t bond_port_id;
+ struct rte_eth_dev_info bond_dev_info;
+ struct rte_eth_rss_reta_entry64 bond_reta_conf[512 / RTE_RETA_GROUP_SIZE];
+ struct slave_conf slave_ports[SLAVE_COUNT];
+
+ struct rte_mempool *mbuf_pool;
+};
+
+static struct link_bonding_rssconf_unittest_params test_params = {
+ .bond_port_id = INVALID_PORT_ID,
+ .slave_ports = {
+ [0 ... SLAVE_COUNT - 1] = { .port_id = INVALID_PORT_ID, .is_slave = 0}
+ },
+ .mbuf_pool = NULL,
+};
+
+/**
+ * Default port configuration with RSS turned off
+ */
+static struct rte_eth_conf default_pmd_conf = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_NONE,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled */
+ .hw_ip_checksum = 0, /**< IP checksum offload enabled */
+ .hw_vlan_filter = 0, /**< VLAN filtering disabled */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled */
+ .hw_strip_crc = 0, /**< CRC stripped by hardware */
+ },
+ .txmode = {
+ .mq_mode = ETH_MQ_TX_NONE,
+ },
+ .lpbk_mode = 0,
+};
+
+static struct rte_eth_conf rss_pmd_conf = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_RSS,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled */
+ .hw_ip_checksum = 0, /**< IP checksum offload enabled */
+ .hw_vlan_filter = 0, /**< VLAN filtering disabled */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled */
+ .hw_strip_crc = 0, /**< CRC stripped by hardware */
+ },
+ .txmode = {
+ .mq_mode = ETH_MQ_TX_NONE,
+ },
+ .rx_adv_conf = {
+ .rss_conf = {
+ .rss_key = NULL,
+ .rss_hf = ETH_RSS_IPV6,
+ },
+ },
+ .lpbk_mode = 0,
+};
+
+#define FOR_EACH(_i, _item, _array, _size) \
+ for (_i = 0, _item = &_array[0]; _i < _size && (_item = &_array[_i]); _i++)
+
+/* Macro for iterating over every port that can be used as a slave
+ * in this test.
+ * _i variable used as an index in test_params->slave_ports
+ * _slave pointer to &test_params->slave_ports[_idx]
+ */
+#define FOR_EACH_PORT(_i, _port) \
+ FOR_EACH(_i, _port, test_params.slave_ports, \
+ RTE_DIM(test_params.slave_ports))
+
+static int
+configure_ethdev(uint8_t port_id, struct rte_eth_conf *eth_conf, uint8_t start)
+{
+ int rxq, txq;
+
+ TEST_ASSERT(rte_eth_dev_configure(port_id, RXTX_QUEUE_COUNT,
+ RXTX_QUEUE_COUNT, eth_conf) == 0, "Failed to configure device %u",
+ port_id);
+
+ for (rxq = 0; rxq < RXTX_QUEUE_COUNT; rxq++) {
+ TEST_ASSERT(rte_eth_rx_queue_setup(port_id, rxq, RXTX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id), NULL,
+ test_params.mbuf_pool) == 0, "Failed to setup rx queue.");
+ }
+
+ for (txq = 0; txq < RXTX_QUEUE_COUNT; txq++) {
+ TEST_ASSERT(rte_eth_tx_queue_setup(port_id, txq, RXTX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id), NULL) == 0,
+ "Failed to setup tx queue.");
+ }
+
+ if (start) {
+ TEST_ASSERT(rte_eth_dev_start(port_id) == 0,
+ "Failed to start device (%d).", port_id);
+ }
+
+ return 0;
+}
+
+/**
+ * Remove all slaves from bonding
+ */
+static int
+remove_slaves(void)
+{
+ unsigned n;
+ struct slave_conf *port;
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ if (port->is_slave) {
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(
+ test_params.bond_port_id, port->port_id),
+ "Cannot remove slave %d from bonding", port->port_id);
+ port->is_slave = 0;
+ }
+ }
+
+ return 0;
+}
+
+static int
+remove_slaves_and_stop_bonded_device(void)
+{
+ TEST_ASSERT_SUCCESS(remove_slaves(), "Removing slaves");
+ rte_eth_dev_stop(test_params.bond_port_id);
+ return TEST_SUCCESS;
+}
+
+/**
+ * Add all slaves to bonding
+ */
+static int
+bond_slaves(void)
+{
+ unsigned n;
+ struct slave_conf *port;
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ if (!port->is_slave) {
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id,
+ port->port_id), "Cannot attach slave %d to the bonding",
+ port->port_id);
+ port->is_slave = 1;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * Set all RETA values in port_id to value
+ */
+static int
+reta_set(uint8_t port_id, uint8_t value, int reta_size)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ int i, j;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) {
+ /* select all fields to set */
+ reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ reta_conf[i].reta[j] = value;
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
+
+/**
+ * Check if slaves RETA is synchronized with bonding port. Returns 1 if slave
+ * port is synced with bonding port.
+ */
+static int
+reta_check_synced(struct slave_conf *port)
+{
+ unsigned i;
+
+ for (i = 0; i < test_params.bond_dev_info.reta_size;
+ i++) {
+
+ int index = i / RTE_RETA_GROUP_SIZE;
+ int shift = i % RTE_RETA_GROUP_SIZE;
+
+ if (port->reta_conf[index].reta[shift] !=
+ test_params.bond_reta_conf[index].reta[shift])
+ return 0;
+
+ }
+
+ return 1;
+}
+
+/**
+ * Fetch bonding ports RETA
+ */
+static int
+bond_reta_fetch(void) {
+ unsigned j;
+
+ for (j = 0; j < test_params.bond_dev_info.reta_size / RTE_RETA_GROUP_SIZE;
+ j++)
+ test_params.bond_reta_conf[j].mask = ~0LL;
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(test_params.bond_port_id,
+ test_params.bond_reta_conf, test_params.bond_dev_info.reta_size),
+ "Cannot take bonding ports RSS configuration");
+ return 0;
+}
+
+/**
+ * Fetch slaves RETA
+ */
+static int
+slave_reta_fetch(struct slave_conf *port) {
+ unsigned j;
+
+ for (j = 0; j < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; j++)
+ port->reta_conf[j].mask = ~0LL;
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(port->port_id,
+ port->reta_conf, port->dev_info.reta_size),
+ "Cannot take bonding ports RSS configuration");
+ return 0;
+}
+
+/**
+ * Remove and add slave to check if slaves configuration is synced with
+ * the bonding ports values after adding new slave.
+ */
+static int
+slave_remove_and_add(void)
+{
+ struct slave_conf *port = &(test_params.slave_ports[0]);
+
+ /* 1. Remove first slave from bonding */
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(test_params.bond_port_id,
+ port->port_id), "Cannot remove slave #d from bonding");
+
+ /* 2. Change removed (ex-)slave and bonding configuration to different
+ * values
+ */
+ reta_set(test_params.bond_port_id, 1, test_params.bond_dev_info.reta_size);
+ bond_reta_fetch();
+
+ reta_set(port->port_id, 2, port->dev_info.reta_size);
+ slave_reta_fetch(port);
+
+ TEST_ASSERT(reta_check_synced(port) == 0,
+ "Removed slave didn't should be synchronized with bonding port");
+
+ /* 3. Add (ex-)slave and check if configuration changed*/
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id,
+ port->port_id), "Cannot add slave");
+
+ bond_reta_fetch();
+ slave_reta_fetch(port);
+
+ return reta_check_synced(port);
+}
+
+/**
+ * Test configuration propagation over slaves.
+ */
+static int
+test_propagate(void)
+{
+ unsigned i;
+ uint8_t n;
+ struct slave_conf *port;
+ uint8_t bond_rss_key[40];
+ struct rte_eth_rss_conf bond_rss_conf;
+
+ int retval = 0;
+ uint64_t rss_hf = 0;
+ uint64_t default_rss_hf = 0;
+
+ rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info);
+
+ /*
+ * Test hash function propagation
+ */
+ for (i = 0; i < sizeof(test_params.bond_dev_info.flow_type_rss_offloads)*8;
+ i++) {
+
+ rss_hf = test_params.bond_dev_info.flow_type_rss_offloads & (1<<i);
+ if (rss_hf) {
+ bond_rss_conf.rss_key = NULL;
+ bond_rss_conf.rss_hf = rss_hf;
+
+ retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id,
+ &bond_rss_conf);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set slaves hash function");
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ retval = rte_eth_dev_rss_hash_conf_get(port->port_id,
+ &port->rss_conf);
+ TEST_ASSERT_SUCCESS(retval,
+ "Cannot take slaves RSS configuration");
+
+ TEST_ASSERT(port->rss_conf.rss_hf == rss_hf,
+ "Hash function not propagated for slave %d",
+ port->port_id);
+ }
+
+ default_rss_hf = rss_hf;
+ }
+
+ }
+
+ /*
+ * Test key propagation
+ */
+ for (i = 1; i < 10; i++) {
+
+ /* Set all keys to zero */
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ memset(port->rss_conf.rss_key, 0, 40);
+ retval = rte_eth_dev_rss_hash_update(port->port_id,
+ &port->rss_conf);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RSS keys");
+ }
+
+ memset(bond_rss_key, i, sizeof(bond_rss_key));
+ bond_rss_conf.rss_hf = default_rss_hf,
+ bond_rss_conf.rss_key = bond_rss_key;
+ bond_rss_conf.rss_key_len = 40;
+
+ retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id,
+ &bond_rss_conf);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set bonded port RSS keys");
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ retval = rte_eth_dev_rss_hash_conf_get(port->port_id,
+ &(port->rss_conf));
+
+ TEST_ASSERT_SUCCESS(retval,
+ "Cannot take slaves RSS configuration");
+
+ /* compare keys */
+ retval = memcmp(port->rss_conf.rss_key, bond_rss_key,
+ sizeof(bond_rss_key));
+ TEST_ASSERT(retval == 0, "Key value not propagated for slave %d",
+ port->port_id);
+ }
+ }
+
+ /*
+ * Test RETA propagation
+ */
+ for (i = 0; i < RXTX_QUEUE_COUNT; i++) {
+
+ /* Set all keys to zero */
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ retval = reta_set(port->port_id, (i + 1) % RXTX_QUEUE_COUNT,
+ port->dev_info.reta_size);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RETA");
+ }
+
+ TEST_ASSERT_SUCCESS(reta_set(test_params.bond_port_id,
+ i % RXTX_QUEUE_COUNT, test_params.bond_dev_info.reta_size),
+ "Cannot set bonded port RETA");
+
+ bond_reta_fetch();
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ slave_reta_fetch(port);
+ TEST_ASSERT(reta_check_synced(port) == 1, "RETAs inconsistent");
+ }
+ }
+
+ return TEST_SUCCESS;
+}
+
+/**
+ * Test propagation logic, when RX_RSS mq_mode is turned on for bonding port
+ */
+static int
+test_rss(void)
+{
+ /**
+ * Configure bonding port in RSS mq mode
+ */
+ TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id,
+ &rss_pmd_conf, 0), "Failed to configure bonding device\n");
+
+ rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info);
+
+ TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed");
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id),
+ "Failed to start bonding port (%d).", test_params.bond_port_id);
+
+ TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed");
+
+ TEST_ASSERT(slave_remove_and_add() == 1, "New slave should be synced");
+
+ remove_slaves_and_stop_bonded_device();
+
+ return TEST_SUCCESS;
+}
+
+/**
+ * Test propagation logic, when RX_RSS mq_mode is turned off for bonding port
+ */
+static int
+test_rss_lazy(void)
+{
+ TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id,
+ &default_pmd_conf, 0), "Failed to configure bonding device\n");
+
+ rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info);
+
+ TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed");
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id),
+ "Failed to start bonding port (%d).", test_params.bond_port_id);
+
+ TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed");
+
+ TEST_ASSERT(slave_remove_and_add() == 0, "New slave shouldn't be synced");
+
+ remove_slaves_and_stop_bonded_device();
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_setup(void)
+{
+ unsigned i, n;
+ int retval;
+ char name[256];
+ char rng_name[RTE_RING_NAMESIZE];
+ struct slave_conf *port;
+
+ if (test_params.mbuf_pool == NULL) {
+
+ test_params.mbuf_pool = rte_mempool_create("MBUF_POOL", NUM_MBUFS *
+ SLAVE_COUNT, MBUF_SIZE, MBUF_CACHE_SIZE,
+ sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init,
+ NULL, rte_pktmbuf_init, NULL, rte_socket_id(), 0);
+
+ TEST_ASSERT(test_params.mbuf_pool != NULL,
+ "rte_mempool_create failed\n");
+ }
+
+ /* Create / initialize ring eth devs. */
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ snprintf(name, sizeof(name), SLAVE_DEV_NAME_FMT, n);
+
+ for (i = 0; i < RXTX_QUEUE_COUNT; i++) {
+ snprintf(rng_name, sizeof(rng_name), SLAVE_RXTX_QUEUE_FMT, n, i);
+
+ if (port->rxtx_queue[i] == NULL) {
+ port->rxtx_queue[i] = rte_ring_create(rng_name, RXTX_RING_SIZE,
+ 0, RING_F_SP_ENQ|RING_F_SC_DEQ);
+ TEST_ASSERT(port->rxtx_queue[i] != NULL,
+ "Failed to allocate rx ring '%s': %s", name,
+ rte_strerror(rte_errno));
+ }
+ }
+
+ if (rte_eth_from_rings(name, port->rxtx_queue, RXTX_QUEUE_COUNT,
+ port->rxtx_queue, RXTX_QUEUE_COUNT, 0)) {
+ TEST_ASSERT(retval >= 0,
+ "Failed to create ring ethdev '%s'\n", name);
+ }
+
+ port->port_id = rte_eth_dev_count() - 1;
+ port->rss_conf.rss_key = port->rss_key;
+ port->rss_conf.rss_key_len = 40;
+
+ retval = configure_ethdev(port->port_id, &default_pmd_conf, 1);
+ TEST_ASSERT_SUCCESS(retval, "Failed to configure virtual ethdev %s\n",
+ name);
+
+ rte_eth_dev_info_get(port->port_id, &port->dev_info);
+ }
+
+ if (test_params.bond_port_id == INVALID_PORT_ID) {
+ retval = rte_eth_bond_create(BONDED_DEV_NAME, 0, rte_socket_id());
+
+ TEST_ASSERT(retval >= 0, "Failed to create bonded ethdev %s",
+ BONDED_DEV_NAME);
+
+ test_params.bond_port_id = retval;
+
+ TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id,
+ &default_pmd_conf, 0), "Failed to configure bonding device\n");
+
+ rte_eth_dev_info_get(test_params.bond_port_id,
+ &test_params.bond_dev_info);
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+check_environment(void)
+{
+ return TEST_SUCCESS;
+}
+
+static int
+test_rssconf_executor(int (*test_func)(void))
+{
+ int test_result;
+
+ /* Check if environment is clean. Fail to launch a test if there was
+ * a critical error before that prevented to reset environment. */
+ TEST_ASSERT_SUCCESS(check_environment(),
+ "Refusing to launch test in dirty environment.");
+
+ RTE_VERIFY(test_func != NULL);
+ test_result = (*test_func)();
+
+ /* If test succeed check if environment wast left in good condition. */
+ if (test_result == TEST_SUCCESS)
+ test_result = check_environment();
+
+ /* Reset environment in case test failed to do that. */
+ if (test_result != TEST_SUCCESS) {
+ TEST_ASSERT_SUCCESS(remove_slaves_and_stop_bonded_device(),
+ "Failed to stop bonded device");
+ }
+
+ return test_result;
+}
+
+static int
+test_setup_wrapper(void)
+{
+ return test_rssconf_executor(&test_setup);
+}
+
+static int
+test_rss_wrapper(void)
+{
+ return test_rssconf_executor(&test_rss);
+}
+
+static int
+test_rss_lazy_wrapper(void)
+{
+ return test_rssconf_executor(&test_rss_lazy);
+}
+
+static struct unit_test_suite link_bonding_rssconf_test_suite = {
+ .suite_name = "RSS Dynamic Configuration for Bonding Unit Test Suite",
+ .setup = test_setup,
+ .unit_test_cases = {
+ TEST_CASE_NAMED("test_setup", test_setup_wrapper),
+ TEST_CASE_NAMED("test_rss", test_rss_wrapper),
+ TEST_CASE_NAMED("test_rss_lazy", test_rss_lazy_wrapper),
+ { NULL, NULL, NULL, NULL, NULL } /**< NULL terminate unit test array */
+ }
+};
+
+static int
+test_link_bonding_rssconf(void)
+{
+ return unit_test_suite_runner(&link_bonding_rssconf_test_suite);
+}
+
+static struct test_command test_link_bonding_rssconf_cmd = {
+ .command = "link_bonding_rssconf_autotest",
+ .callback = test_link_bonding_rssconf,
+};
+
+REGISTER_TEST_COMMAND(test_link_bonding_rssconf_cmd);
--
1.7.9.5
Tomasz Kulasek
2015-06-29 14:50:40 UTC
Permalink
Queue stats mapping is used in example application. This patch adds
propagation of mapping over the slaves, and fills bonding port's stats
with a sum of corresponding values taken from bonded slaves, when stats
are requested for bonding port.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/bonding/rte_eth_bond_pmd.c | 30 +++++++++++++++++++++++++++++-
1 file changed, 29 insertions(+), 1 deletion(-)

diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index cd23f42..ade84c4 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -1783,7 +1783,7 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
{
struct bond_dev_private *internals = dev->data->dev_private;
struct rte_eth_stats slave_stats;
- int i;
+ int i, j;

for (i = 0; i < internals->slave_count; i++) {
rte_eth_stats_get(internals->slaves[i].port_id, &slave_stats);
@@ -1802,6 +1802,15 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
stats->rx_pause_xon += slave_stats.rx_pause_xon;
stats->tx_pause_xoff += slave_stats.tx_pause_xoff;
stats->rx_pause_xoff += slave_stats.rx_pause_xoff;
+
+ for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) {
+ stats->q_ipackets[j] += slave_stats.q_ipackets[j];
+ stats->q_opackets[j] += slave_stats.q_ipackets[j];
+ stats->q_ibytes[j] += slave_stats.q_ipackets[j];
+ stats->q_obytes[j] += slave_stats.q_ipackets[j];
+ stats->q_errors[j] += slave_stats.q_ipackets[j];
+ }
+
}
}

@@ -2110,6 +2119,24 @@ bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev,
return 0;
}

+static int
+bond_ethdev_queue_stats_mapping_set(struct rte_eth_dev *dev, uint16_t queue_id,
+ uint8_t stat_idx, uint8_t is_rx)
+{
+ int i;
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ for (i = 0; i < internals->slave_count; i++)
+ if (is_rx)
+ rte_eth_dev_set_rx_queue_stats_mapping(
+ internals->slaves[i].port_id, queue_id, stat_idx);
+ else
+ rte_eth_dev_set_tx_queue_stats_mapping(
+ internals->slaves[i].port_id, queue_id, stat_idx);
+
+ return 0;
+}
+
struct eth_dev_ops default_dev_ops = {
.dev_start = bond_ethdev_start,
.dev_stop = bond_ethdev_stop,
@@ -2123,6 +2150,7 @@ struct eth_dev_ops default_dev_ops = {
.link_update = bond_ethdev_link_update,
.stats_get = bond_ethdev_stats_get,
.stats_reset = bond_ethdev_stats_reset,
+ .queue_stats_mapping_set = bond_ethdev_queue_stats_mapping_set,
.promiscuous_enable = bond_ethdev_promiscuous_enable,
.promiscuous_disable = bond_ethdev_promiscuous_disable,
.reta_update = bond_ethdev_rss_reta_update,
--
1.7.9.5
Thomas Monjalon
2015-07-13 11:18:12 UTC
Permalink
Post by Tomasz Kulasek
+ for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) {
+ stats->q_ipackets[j] += slave_stats.q_ipackets[j];
+ stats->q_opackets[j] += slave_stats.q_ipackets[j];
+ stats->q_ibytes[j] += slave_stats.q_ipackets[j];
+ stats->q_obytes[j] += slave_stats.q_ipackets[j];
+ stats->q_errors[j] += slave_stats.q_ipackets[j];
Wrong copy/paste.
Kulasek, TomaszX
2015-07-13 12:00:51 UTC
Permalink
-----Original Message-----
Sent: Monday, July 13, 2015 13:18
To: Kulasek, TomaszX
Subject: Re: [dpdk-dev] [PATCH v3 4/8] bonding: queue stats mapping
Post by Tomasz Kulasek
+ for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) {
+ stats->q_ipackets[j] += slave_stats.q_ipackets[j];
+ stats->q_opackets[j] += slave_stats.q_ipackets[j];
+ stats->q_ibytes[j] += slave_stats.q_ipackets[j];
+ stats->q_obytes[j] += slave_stats.q_ipackets[j];
+ stats->q_errors[j] += slave_stats.q_ipackets[j];
Wrong copy/paste.
Ok, my fault, sorry. I'll fix it.
Tomasz Kulasek
2015-06-29 14:50:41 UTC
Permalink
Per queue statistics are already implemented for ring device, but with
static mapping (stat_idx == queue_id).

This fix is required, if you want to use ring device in test application
and is used only to point that per queue statistics are provided for this
device.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/ring/rte_eth_ring.c | 10 ++++++++++
1 file changed, 10 insertions(+)

diff --git a/drivers/net/ring/rte_eth_ring.c b/drivers/net/ring/rte_eth_ring.c
index 29d7bd8..7e6bf14 100644
--- a/drivers/net/ring/rte_eth_ring.c
+++ b/drivers/net/ring/rte_eth_ring.c
@@ -333,6 +333,15 @@ eth_rss_hash_conf_get(struct rte_eth_dev *dev,
return 0;
}

+static int
+eth_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, __rte_unused uint8_t stat_idx,
+ __rte_unused uint8_t is_rx)
+{
+ /* Do nothing, just return ok */
+ return 0;
+}
+
static const struct eth_dev_ops ops = {
.dev_start = eth_dev_start,
.dev_stop = eth_dev_stop,
@@ -345,6 +354,7 @@ static const struct eth_dev_ops ops = {
.rx_queue_release = eth_queue_release,
.tx_queue_release = eth_queue_release,
.link_update = eth_link_update,
+ .queue_stats_mapping_set = eth_queue_stats_mapping_set,
.stats_get = eth_stats_get,
.stats_reset = eth_stats_reset,
.mac_addr_remove = eth_mac_addr_remove,
--
1.7.9.5
Thomas Monjalon
2015-07-09 01:58:02 UTC
Permalink
Post by Tomasz Kulasek
Per queue statistics are already implemented for ring device, but with
static mapping (stat_idx == queue_id).
This fix is required, if you want to use ring device in test application
and is used only to point that per queue statistics are provided for this
device.
[...]
Post by Tomasz Kulasek
+static int
+eth_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, __rte_unused uint8_t stat_idx,
+ __rte_unused uint8_t is_rx)
+{
+ /* Do nothing, just return ok */
+ return 0;
+}
I've just realized how this is broken.
Some Intel devices use a mapping to select hardware queues which will have
some stats. But we may have stats per queues without requiring such mapping.

I may miss something but I suggest these 3 actions:
- remove this patch
- replace checks on stats_mapping by an ethdev flag
- remove device-specific stats_mapping from ethdev
Kulasek, TomaszX
2015-07-09 09:55:10 UTC
Permalink
Sent: Thursday, July 9, 2015 03:58
To: Kulasek, TomaszX
Subject: Re: [dpdk-dev] [PATCH v3 5/8] ring: queue stats mapping set dummy
implementation
Post by Tomasz Kulasek
Per queue statistics are already implemented for ring device, but with
static mapping (stat_idx == queue_id).
This fix is required, if you want to use ring device in test
application and is used only to point that per queue statistics are
provided for this device.
[...]
Post by Tomasz Kulasek
+static int
+eth_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, __rte_unused uint8_t stat_idx,
+ __rte_unused uint8_t is_rx)
+{
+ /* Do nothing, just return ok */
+ return 0;
+}
I've just realized how this is broken.
Some Intel devices use a mapping to select hardware queues which will have
some stats. But we may have stats per queues without requiring such mapping.
- remove this patch
- replace checks on stats_mapping by an ethdev flag
- remove device-specific stats_mapping from ethdev
For Niantic NICs all queues stats for port are mapped to stats_idx=0 by default, and stats mapping is required to have per-queue statistics (even in testpmd).

Anyway, this patch, for ring pmd, was intended more as a cleanup than feature and was inspired by implementation in virtio driver:

virtio_ethdev.c:

/*
* It enables testpmd to collect per queue stats.
*/
static int
virtio_dev_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *eth_dev,
__rte_unused uint16_t queue_id, __rte_unused uint8_t stat_idx,
__rte_unused uint8_t is_rx)
{
return 0;
}


This patch can be safely removed, if you think such a cleanup is not required, or lack of this implementation should be common behavior for this case. That will cause only few more warning messages, if you want to use ring pmd as a slave in example application.

Do you need v4?
Thomas Monjalon
2015-07-09 10:13:56 UTC
Permalink
Post by Kulasek, TomaszX
Sent: Thursday, July 9, 2015 03:58
To: Kulasek, TomaszX
Subject: Re: [dpdk-dev] [PATCH v3 5/8] ring: queue stats mapping set dummy
implementation
Post by Tomasz Kulasek
Per queue statistics are already implemented for ring device, but with
static mapping (stat_idx == queue_id).
This fix is required, if you want to use ring device in test
application and is used only to point that per queue statistics are
provided for this device.
[...]
Post by Tomasz Kulasek
+static int
+eth_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *dev,
+ __rte_unused uint16_t queue_id, __rte_unused uint8_t stat_idx,
+ __rte_unused uint8_t is_rx)
+{
+ /* Do nothing, just return ok */
+ return 0;
+}
I've just realized how this is broken.
Some Intel devices use a mapping to select hardware queues which will have
some stats. But we may have stats per queues without requiring such mapping.
- remove this patch
- replace checks on stats_mapping by an ethdev flag
- remove device-specific stats_mapping from ethdev
For Niantic NICs all queues stats for port are mapped to stats_idx=0 by default, and stats mapping is required to have per-queue statistics (even in testpmd).
/*
* It enables testpmd to collect per queue stats.
*/
It should be removed in virtio.
Post by Kulasek, TomaszX
static int
virtio_dev_queue_stats_mapping_set(__rte_unused struct rte_eth_dev *eth_dev,
__rte_unused uint16_t queue_id, __rte_unused uint8_t stat_idx,
__rte_unused uint8_t is_rx)
{
return 0;
}
This patch can be safely removed, if you think such a cleanup is not required,
OK
Post by Kulasek, TomaszX
or lack of this implementation should be common behavior for this case.
That will cause only few more warning messages, if you want to use ring pmd
as a slave in example application.
Do you need v4?
No, thank you
Tomasz Kulasek
2015-06-29 14:50:42 UTC
Permalink
This application allows you to test RSS configuration for bonded devices
changing configuration dynamically, during receiving packets on slaves and
observe the changes in its distribution over queues.

After initialization process, all accessible ports are attached to one
bonding as slaves.

Monitor screen is divided into five main parts:
- Port selection (on the very top)
- RSS Configuration for selected port including hash function, key and
RETA
- Incoming packets statistics
- Incoming packets list for selected port
- Status bar with contextual information about selected part

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
examples/bond_rss/Makefile | 59 +++
examples/bond_rss/bondrss.c | 293 ++++++++++++++
examples/bond_rss/bondrss.h | 163 ++++++++
examples/bond_rss/config.c | 251 ++++++++++++
examples/bond_rss/ui.c | 945 +++++++++++++++++++++++++++++++++++++++++++
5 files changed, 1711 insertions(+)
create mode 100644 examples/bond_rss/Makefile
create mode 100644 examples/bond_rss/bondrss.c
create mode 100644 examples/bond_rss/bondrss.h
create mode 100644 examples/bond_rss/config.c
create mode 100644 examples/bond_rss/ui.c

diff --git a/examples/bond_rss/Makefile b/examples/bond_rss/Makefile
new file mode 100644
index 0000000..db457a3
--- /dev/null
+++ b/examples/bond_rss/Makefile
@@ -0,0 +1,59 @@
+# BSD LICENSE
+#
+# Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+# All rights reserved.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+#
+# * Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# * Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in
+# the documentation and/or other materials provided with the
+# distribution.
+# * Neither the name of Intel Corporation nor the names of its
+# contributors may be used to endorse or promote products derived
+# from this software without specific prior written permission.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+ifeq ($(RTE_SDK),)
+$(error "Please define RTE_SDK environment variable")
+endif
+
+# Default target, can be overridden by command line or environment
+RTE_TARGET ?= x86_64-native-linuxapp-gcc
+
+include $(RTE_SDK)/mk/rte.vars.mk
+
+# binary name
+APP = bondrss
+
+# all source are stored in SRCS-y
+SRCS-y := bondrss.c
+SRCS-y += ui.c
+SRCS-y += config.c
+
+CFLAGS += $(WERROR_FLAGS)
+
+# workaround for a gcc bug with noreturn attribute
+# http://gcc.gnu.org/bugzilla/show_bug.cgi?id=12603
+ifeq ($(CONFIG_RTE_TOOLCHAIN_GCC),y)
+CFLAGS_main.o += -Wno-return-type
+endif
+
+LDLIBS += -lncurses
+
+include $(RTE_SDK)/mk/rte.extapp.mk
diff --git a/examples/bond_rss/bondrss.c b/examples/bond_rss/bondrss.c
new file mode 100644
index 0000000..225bf24
--- /dev/null
+++ b/examples/bond_rss/bondrss.c
@@ -0,0 +1,293 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include "bondrss.h"
+
+#define RSS_HASH_KEY_LENGTH 40
+
+static const struct rte_eth_conf port_conf_default = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_RSS,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled. */
+ .hw_ip_checksum = 0, /**< IP checksum offload disabled. */
+ .hw_vlan_filter = 1, /**< VLAN filtering enabled. */
+ .hw_vlan_strip = 1, /**< VLAN strip enabled. */
+ .hw_vlan_extend = 0, /**< Extended VLAN disabled. */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */
+ .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */
+ .enable_scatter = 0, /**< scatter rx disabled */
+ },
+ .rx_adv_conf = {
+ .rss_conf = {
+ .rss_key = NULL,
+ .rss_hf = ETH_RSS_IPV4,
+ },
+ },
+};
+
+static const struct rte_eth_conf port_conf_bonding = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_RSS,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled. */
+ .hw_ip_checksum = 0, /**< IP checksum offload disabled. */
+ .hw_vlan_filter = 1, /**< VLAN filtering enabled. */
+ .hw_vlan_strip = 1, /**< VLAN strip enabled. */
+ .hw_vlan_extend = 0, /**< Extended VLAN disabled. */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */
+ .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */
+ .enable_scatter = 0, /**< scatter rx disabled */
+ },
+ .rx_adv_conf = {
+ .rss_conf = {
+ .rss_key = NULL,
+ .rss_hf = ETH_RSS_IPV6,
+ },
+ },
+};
+
+/*
+ * Configurable number of RX/TX queues.
+ */
+int nb_rxq = 4; /**< Number of RX queues per port. */
+int nb_txq = 1; /**< Number of TX queues per port. */
+
+int bond_port_id = -1; /**< Bonded device port id */
+
+int nb_ports;
+struct rte_port *ports;
+rte_atomic64_t counter; /**< Received packet's counter */
+
+/*
+ * Initializes a given port using global settings and with the rx buffers
+ * coming from the mbuf_pool passed as parameter
+ */
+static inline
+int init_port(uint8_t port_id, struct rte_mempool *mbuf_pool,
+ struct rte_eth_conf port_conf)
+{
+
+ int retval;
+ uint16_t q;
+ struct rte_eth_rxconf rxconf;
+
+ struct rte_port *port;
+
+ if (port_id >= rte_eth_dev_count())
+ return -1;
+
+ port = &ports[port_id];
+ port->nb_rx_queues = nb_rxq;
+ port->nb_tx_queues = nb_txq;
+
+ memcpy(&(port->dev_conf), &port_conf, sizeof(port_conf));
+ retval = rte_eth_dev_configure(port_id, port->nb_rx_queues,
+ port->nb_tx_queues, &port_conf);
+
+ if (retval != 0)
+ return retval;
+
+ rte_eth_dev_info_get(port_id, &(port->dev_info));
+ rxconf = (port->dev_info).default_rxconf;
+ rxconf.rx_free_thresh = 32;
+
+ for (q = 0; q < port->nb_rx_queues; q++) {
+ retval = rte_eth_rx_queue_setup(port_id, q, RX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id), &rxconf, mbuf_pool);
+ if (retval < 0)
+ return retval;
+ }
+
+ for (q = 0; q < port->nb_tx_queues; q++) {
+ retval = rte_eth_tx_queue_setup(port_id, q, TX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id),
+ &((port->dev_info).default_txconf));
+ if (retval < 0)
+ return retval;
+ }
+
+ port->bond_mode = -1;
+ port->promiscuous = 0;
+ port->enabled = 1;
+
+ return 0;
+}
+
+static int
+rx_loop(__attribute__((unused)) void *dummy)
+{
+ uint8_t port_id;
+ int nq;
+ int n;
+
+ for (;;)
+ for (port_id = 0; port_id < nb_ports; port_id++) {
+ /* Pool only bonding ports */
+ if (ports[port_id].bond_mode >= 0)
+ for (nq = 0; nq < ports[port_id].nb_rx_queues; nq++) {
+ struct rte_mbuf *bufs[BURST_SIZE];
+
+ const uint16_t nb_rx = rte_eth_rx_burst(port_id, nq, bufs,
+ BURST_SIZE);
+ if (unlikely(nb_rx == 0))
+ continue;
+
+ ports[port_id].rxq_ipackets[nq] += nb_rx;
+ for (n = 0; n < nb_rx; n++) {
+ record_pkt(port_id, nq, bufs[n]);
+ rte_pktmbuf_free(bufs[n]);
+ }
+ }
+ }
+
+ return 0;
+}
+
+static uint16_t
+rx_callback(uint8_t port_id, uint16_t qidx, struct rte_mbuf **pkts,
+ uint16_t nb_pkts, uint16_t max_pkts __rte_unused, void *_ __rte_unused)
+{
+ int n;
+
+ ports[port_id].rxq_ipackets[qidx] += nb_pkts;
+ for (n = 0; n < nb_pkts; n++) {
+ rte_atomic64_inc(&counter);
+ pkts[n]->udata64 = counter.cnt;
+ record_pkt(port_id, qidx, pkts[n]);
+ }
+
+ return nb_pkts;
+}
+
+int
+main(int argc, char *argv[])
+{
+ struct rte_mempool *mbuf_pool;
+ unsigned lcore_id = 0;
+ uint8_t port_id, queue_id;
+
+ /* init EAL */
+ int ret = rte_eal_init(argc, argv);
+
+ if (ret < 0)
+ rte_exit(EXIT_FAILURE, "Error with EAL initialization\n");
+ argc -= ret;
+ argv += ret;
+
+ nb_ports = rte_eth_dev_count();
+ if (nb_ports < 1)
+ rte_exit(EXIT_FAILURE,
+ "At least one port must be available to create a bonding\n");
+
+ mbuf_pool = rte_mempool_create("MBUF_POOL", NUM_MBUFS * nb_ports, MBUF_SIZE,
+ MBUF_CACHE_SIZE, sizeof(struct rte_pktmbuf_pool_private),
+ rte_pktmbuf_pool_init, NULL, rte_pktmbuf_init, NULL,
+ rte_socket_id(), 0);
+
+ if (mbuf_pool == NULL)
+ rte_exit(EXIT_FAILURE, "Cannot create mbuf pool\n");
+
+ /* Configuration of Ethernet ports. */
+ ports = rte_zmalloc("bondrss: ports",
+ sizeof(struct rte_port) * RTE_MAX_ETHPORTS, RTE_CACHE_LINE_SIZE);
+
+ if (ports == NULL)
+ rte_exit(EXIT_FAILURE, "rte_zmalloc(%d struct rte_port) failed\n",
+ RTE_MAX_ETHPORTS);
+
+ /* enabled allocated ports */
+ for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++)
+ ports[port_id].enabled = 0;
+
+ /* initialize all ports */
+ for (port_id = 0; port_id < nb_ports; port_id++) {
+ if (init_port(port_id, mbuf_pool, port_conf_default) != 0)
+ rte_exit(EXIT_FAILURE, "Cannot init port %"PRIu8"\n", port_id);
+
+ /* Add rx callbacks to the slave's port */
+ for (queue_id = 0; queue_id < ports[port_id].nb_rx_queues; queue_id++)
+ rte_eth_add_rx_callback(port_id, queue_id, rx_callback, NULL);
+
+ }
+
+ /* create bonding port */
+ bond_port_id = rte_eth_bond_create("eth_bond", 0, 0);
+ if (bond_port_id < 0)
+ rte_exit(EXIT_FAILURE, "Cannot create bonding port\n");
+
+ for (port_id = 0; port_id < nb_ports; port_id++)
+ rte_eth_bond_slave_add(bond_port_id, port_id);
+
+ /* count again */
+ nb_ports = rte_eth_dev_count();
+
+ init_port(bond_port_id, mbuf_pool, port_conf_bonding);
+
+ /* start bonding port*/
+ ret = rte_eth_dev_start(bond_port_id);
+
+ /* enable promiscuous by default */
+ rte_eth_promiscuous_enable(bond_port_id);
+
+ for (port_id = 0; port_id < nb_ports; port_id++) {
+ ports[port_id].bond_mode = rte_eth_bond_mode_get(port_id);
+ ports[port_id].promiscuous = rte_eth_promiscuous_get(port_id);
+
+ /* map queues to stats */
+ if (ports[port_id].bond_mode >= 0) {
+ for (queue_id = 0; queue_id < ports[port_id].nb_rx_queues;
+ queue_id++) {
+ rte_eth_dev_set_rx_queue_stats_mapping(port_id, queue_id,
+ queue_id);
+ }
+ }
+
+ }
+
+ /* Initialize received packet's counter */
+ rte_atomic64_init(&counter);
+
+ if (rte_lcore_count() <= 1)
+ rte_exit(EXIT_FAILURE, "More than one free lcore is needed\n");
+
+ lcore_id = rte_get_next_lcore(rte_lcore_id(), 1, 0);
+ rte_eal_remote_launch(rx_loop, NULL, lcore_id);
+
+ /* call lcore_main on master core only */
+ ui_loop();
+
+ return 0;
+}
diff --git a/examples/bond_rss/bondrss.h b/examples/bond_rss/bondrss.h
new file mode 100644
index 0000000..305a690
--- /dev/null
+++ b/examples/bond_rss/bondrss.h
@@ -0,0 +1,163 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef BONDRSS_H_
+#define BONDRSS_H_
+
+#include <stdint.h>
+#include <inttypes.h>
+#include <rte_eal.h>
+#include <rte_ethdev.h>
+#include <rte_cycles.h>
+#include <rte_lcore.h>
+#include <rte_mbuf.h>
+#include <rte_malloc.h>
+
+#include <rte_eth_bond.h>
+
+#define RX_RING_SIZE 128
+#define TX_RING_SIZE 512
+
+#define NUM_MBUFS 8191
+#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)
+#define MBUF_CACHE_SIZE 250
+#define BURST_SIZE 32
+
+/*
+ * Configurable number of RX/TX queues.
+ */
+extern int nb_rxq; /**< Number of RX queues per port. */
+extern int nb_txq; /**< Number of TX queues per port. */
+
+extern int bond_port_id; /**< Bonded device port id */
+extern int bond_mode;
+
+struct rss_type_info {
+ char str[32];
+ uint64_t rss_type;
+};
+
+static struct rss_type_info rss_type_table[] = {
+ {"ipv4", ETH_RSS_IPV4},
+ {"ipv4-frag", ETH_RSS_FRAG_IPV4},
+ {"ipv4-tcp", ETH_RSS_NONFRAG_IPV4_TCP},
+ {"ipv4-udp", ETH_RSS_NONFRAG_IPV4_UDP},
+ {"ipv4-sctp", ETH_RSS_NONFRAG_IPV4_SCTP},
+ {"ipv4-other", ETH_RSS_NONFRAG_IPV4_OTHER},
+ {"ipv6", ETH_RSS_IPV6},
+ {"ipv6-frag", ETH_RSS_FRAG_IPV6},
+ {"ipv6-tcp", ETH_RSS_NONFRAG_IPV6_TCP},
+ {"ipv6-udp", ETH_RSS_NONFRAG_IPV6_UDP},
+ {"ipv6-sctp", ETH_RSS_NONFRAG_IPV6_SCTP},
+ {"ipv6-other", ETH_RSS_NONFRAG_IPV6_OTHER},
+ {"l2-payload", ETH_RSS_L2_PAYLOAD},
+ {"ipv6-ex", ETH_RSS_IPV6_EX},
+ {"ipv6-tcp-ex", ETH_RSS_IPV6_TCP_EX},
+ {"ipv6-udp-ex", ETH_RSS_IPV6_UDP_EX},
+};
+
+struct rss_type_fn {
+ uint8_t enabled;
+ struct rss_type_info *info;
+};
+
+/**
+ * Buffer for last received packets
+ */
+#define PKT_RECORD_SIZE 256
+
+struct pkt_info {
+ uint64_t n; /**< Packet number */
+ uint8_t port_id;
+ uint8_t queue_id;
+ uint64_t ol_flags; /**< Offload features. */
+ uint32_t rss; /**< RSS hash result if RSS enabled */
+};
+
+struct pkt_record {
+ uint64_t count; /**< Total number of received packets */
+ int top;
+ struct pkt_info pkt_info[PKT_RECORD_SIZE];
+};
+
+/**
+ * The data structure associated with each port.
+ */
+struct rte_port {
+ struct rte_eth_dev_info dev_info; /**< PCI info + driver name */
+ struct rte_eth_conf dev_conf; /**< Port configuration. */
+
+ uint16_t nb_rx_queues; /**< Total number of rx queues */
+ uint16_t nb_tx_queues; /**< Total number of tx queues*/
+
+ int bond_mode;
+ struct ether_addr addr;
+
+ int rss_type_count;
+ struct rss_type_fn rss_type_fn[RTE_DIM(rss_type_table)];
+ struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE];
+ uint8_t rss_key[52];
+
+
+ struct rte_eth_stats stats;
+ uint64_t rxq_ipackets[128];
+
+ struct pkt_record record;
+
+ uint8_t rss; /**< RSS enabled for port */
+ uint8_t promiscuous; /**< promiscuous mode enabled for port */
+ uint8_t enabled; /**< port is enabled */
+};
+
+extern int nb_ports;
+extern struct rte_port *ports;
+extern rte_atomic64_t counter; /**< Received packet's counter */
+
+void update_port_info(uint8_t port_id);
+
+int rss_enable(uint8_t port_id, int enabled);
+
+int key_set(uint8_t port_id, uint8_t *key, uint8_t len);
+int key_set_random(uint8_t port_id, uint8_t len);
+
+int reta_set_default(uint8_t port_id);
+int reta_set_random(uint8_t port_id);
+int reta_set_all(uint8_t port_id, uint8_t value);
+int reta_set_weights(uint8_t port_id, uint64_t *weights);
+
+struct pkt_info *record_pkt(uint8_t port_id, int queue_id, struct rte_mbuf *mb);
+
+void ui_loop(void);
+
+
+#endif /* BONDRSS_H_ */
diff --git a/examples/bond_rss/config.c b/examples/bond_rss/config.c
new file mode 100644
index 0000000..fc10d2b
--- /dev/null
+++ b/examples/bond_rss/config.c
@@ -0,0 +1,251 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <time.h>
+#include <stdlib.h>
+#include "bondrss.h"
+
+static const struct rte_eth_conf port_conf_default = {
+ .rxmode = {
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled. */
+ .hw_ip_checksum = 0, /**< IP checksum offload disabled. */
+ .hw_vlan_filter = 1, /**< VLAN filtering enabled. */
+ .hw_vlan_strip = 1, /**< VLAN strip enabled. */
+ .hw_vlan_extend = 0, /**< Extended VLAN disabled. */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled. */
+ .hw_strip_crc = 0, /**< CRC stripping by hardware disabled. */
+ .enable_scatter = 0, /**< scatter rx disabled */
+ },
+ .rx_adv_conf = {
+ .rss_conf = {
+ .rss_key = NULL,
+ .rss_hf = ETH_RSS_IPV4,
+ },
+ },
+};
+
+
+void
+update_port_info(uint8_t port_id)
+{
+ int i;
+ struct rte_port *port;
+
+ port = &ports[port_id];
+
+ rte_eth_dev_info_get(port_id, &(port->dev_info));
+
+ port->promiscuous = rte_eth_promiscuous_get(port_id);
+
+ /* Update device information */
+ rte_eth_dev_info_get(port_id, &port->dev_info);
+ rte_eth_macaddr_get(port_id, &port->addr);
+
+ port->dev_conf.rx_adv_conf.rss_conf.rss_key = port->rss_key;
+ port->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 40;
+ rte_eth_dev_rss_hash_conf_get(port_id,
+ &(port->dev_conf.rx_adv_conf.rss_conf));
+
+ /* select all fields to be fetched */
+ for (i = 0; i < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; i++)
+ port->reta_conf[i].mask = ~0LL;
+
+ rte_eth_dev_rss_reta_query(port_id, port->reta_conf,
+ port->dev_info.reta_size);
+
+}
+
+/**
+ * Try to enable RSS for port_id.
+ */
+int
+rss_enable(uint8_t port_id, int enabled)
+{
+ struct rte_port *port;
+ int retval;
+
+ port = &ports[port_id];
+
+ rte_eth_dev_stop(port_id);
+
+ if (enabled) {
+ port->dev_conf.rxmode.mq_mode = ETH_MQ_RX_RSS;
+ port->dev_conf.rx_adv_conf.rss_conf.rss_hf = ETH_RSS_IPV4;
+ } else {
+ port->dev_conf.rxmode.mq_mode = 0;
+ port->dev_conf.rx_adv_conf.rss_conf.rss_hf = 0;
+ }
+
+ retval = rte_eth_dev_configure(port_id, port->nb_rx_queues,
+ port->nb_tx_queues, &port->dev_conf);
+ if (retval != 0)
+ return retval;
+
+ retval = rte_eth_dev_start(port_id);
+
+ return 0;
+}
+
+int
+key_set(uint8_t port_id, uint8_t *key, uint8_t len)
+{
+ struct rte_eth_rss_conf rss_conf;
+
+ int i;
+
+ rss_conf.rss_key = NULL;
+ rss_conf.rss_key_len = 0;
+ i = rte_eth_dev_rss_hash_conf_get(port_id, &rss_conf);
+
+ rss_conf.rss_key = key;
+ rss_conf.rss_key_len = len;
+
+ i = rte_eth_dev_rss_hash_update(port_id,
+ &rss_conf);
+
+ return i;
+}
+
+/**
+ * Set random RSS key value
+ */
+int
+key_set_random(uint8_t port_id, uint8_t len)
+{
+ int i;
+ uint8_t key[40];
+
+ for (i = 0; i < len; i++)
+ key[i] = rand() % 256;
+
+ return key_set(port_id, key, len);
+}
+
+/**
+ * Set random RETA values
+ */
+int
+reta_set_random(uint8_t port_id)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ int i, j;
+
+ int reta_size = ports[port_id].dev_info.reta_size;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) {
+ /* select all fields to set */
+ reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ reta_conf[i].reta[j] = rand() % ports[port_id].nb_rx_queues;
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
+
+/**
+ * Set default RETA values
+ */
+int
+reta_set_default(uint8_t port_id)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ int i, j;
+
+ int reta_size = ports[port_id].dev_info.reta_size;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) {
+ /* select all fields to set */
+ reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ reta_conf[i].reta[j] = j % nb_rxq;
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
+
+/**
+ * Fill the RETA with values
+ */
+int
+reta_set_all(uint8_t port_id, uint8_t value)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ int i, j;
+
+ int reta_size = ports[port_id].dev_info.reta_size;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) {
+ /* select all fields to set */
+ reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ reta_conf[i].reta[j] = value;
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
+
+/**
+ * Set the RETA basing on weights table
+ */
+int
+reta_set_weights(uint8_t port_id, uint64_t *weights)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ unsigned i, j, k;
+
+ unsigned reta_size = ports[port_id].dev_info.reta_size;
+
+ uint64_t sum = 0, sum2;
+
+ for (i = 0; i < ports[port_id].nb_rx_queues; i++)
+ sum += weights[i];
+ sum2 = sum * (ports[port_id].nb_rx_queues-1);
+
+ if (sum2 == 0)
+ return 0;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++)
+ reta_conf[i].mask = ~0LL;
+ k = 0;
+ for (i = 0; i < ports[port_id].nb_rx_queues; i++) {
+ for (j = 0; k < reta_size &&
+ j < ((sum - weights[i]) * reta_size + 1) / sum2; j++) {
+ reta_conf[k/RTE_RETA_GROUP_SIZE].reta[k%RTE_RETA_GROUP_SIZE] = i;
+ k++;
+ }
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
diff --git a/examples/bond_rss/ui.c b/examples/bond_rss/ui.c
new file mode 100644
index 0000000..d2babe5
--- /dev/null
+++ b/examples/bond_rss/ui.c
@@ -0,0 +1,945 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <rte_atomic.h>
+
+#include <ncurses.h>
+#include <string.h>
+#include <signal.h>
+#include <sys/ioctl.h>
+#include <menu.h>
+#include "bondrss.h"
+
+WINDOW *win_footer;
+WINDOW *win_header;
+WINDOW *win_list;
+WINDOW *win_stats;
+WINDOW *win_reta;
+WINDOW *win_err;
+
+int scr_lines;
+int scr_cols;
+
+unsigned win_header_lines;
+unsigned win_list_lines;
+unsigned win_stats_lines;
+unsigned win_reta_lines;
+unsigned win_err_lines;
+
+const char *ui_status_text;
+static char *ui_err_text[5];
+
+struct rte_port *selected_port;
+static uint8_t selected_port_id;
+
+struct rte_port *focused_port;
+static uint8_t focused_port_id;
+
+#define UI_MODE_PORT 0
+#define UI_MODE_RSS_FN 1
+#define UI_MODE_RSS_KEY 2
+#define UI_MODE_RETA 3
+#define UI_MODE_HELP 4
+
+int ui_mode = UI_MODE_PORT;
+
+static uint16_t _is_lock;
+static uint16_t _is_ready;
+
+static uint16_t _do_configure;
+static uint16_t _do_repaint;
+static uint16_t _do_repaint_stats;
+static uint16_t _do_repaint_list;
+static uint16_t _do_repaint_info;
+static uint16_t _do_repaint_err;
+
+
+struct help_page {
+ const char *title;
+ const char *body;
+};
+
+static struct help_page help_pages[3] = {
+ {
+ .title = "About",
+
+ .body =
+ "\n\n"
+ " ______ ______ ______ _____ ______ ______ ______\n"
+ "| | | \\ / | | \\ | | \\ \\ | | \\ \\ | | | \\ / | / |\n"
+ "| |--| < | | | | | | | | | | | | | |__| | '------. '------.\n"
+ "|_|__|_/ \\_|__|_/ |_| |_| |_|_/_/ |_| \\_\\ ____|_/ ____|_/\n"
+ "\n\n"
+ "This application allows you to test RSS configuration for bonded devices\n"
+ "changing configuration dynamically, during receiving packets on slaves and\n"
+ "observe the changes in its distribution over queues.\n"
+ "\n"
+
+ },
+ {
+ .title = "Introduction",
+ .body = "\n\n"
+ "After initialization process, all accessible ports are attached to one\n"
+ "bonding as slaves.\n"
+ "\n"
+
+ "Monitor screen is divided into five main parts:\n"
+ " - Port selection (on the very top)\n"
+ " - RSS Configuration for selected port including hash function, key and\n"
+ " RETA\n"
+ " - Incoming packets statistics\n"
+ " - Incoming packets list for selected port\n"
+ " - Status bar with contextual information about selected part\n"
+ "\n\n"
+
+ "[up], [down] arrows key lets you highlight a part and parameter to change."
+ },
+ {
+ .title = "Key bindings",
+ .body =
+ "_Port_selection_\n\n"
+ " [left], [right] - select port,\n"
+ " [space] - force turning on/off RSS mode,\n"
+ " [p] - turn on/off promiscuous mode,\n"
+ " [c] - clear statistics and incoming packet list,\n"
+ "\n"
+ "_RSS_hash_function_\n\n"
+ " [1]-[9] - toggle selected hash function,\n"
+ " [a] - select all,\n"
+ "\n"
+ "_RSS_key_\n\n"
+ " [1]-[4] - set one of predefined key value,\n"
+ " [r] - randomize value,\n"
+ "\n"
+ "_RSS_RETA_\n\n"
+ " [number] - fill RETA with number,\n"
+ " [r] - randomize value,\n"
+ " [d] - set default value,"
+
+ }
+};
+
+static int
+ui_lock(int wait)
+{
+ int result;
+
+ while (!(result = rte_atomic16_cmpset(&_is_lock, 0, 1)) && wait)
+ rte_pause();
+
+ return result;
+}
+
+static int
+ui_unlock(void)
+{
+ return rte_atomic16_cmpset(&_is_lock, 1, 0);
+}
+
+/**
+ * Predefined key values
+ */
+static uint8_t RSS_KEY[4][40] = {
+ { 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D,
+ 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+ 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B,
+ 0xBE, 0xAC, 0x01, 0xFA
+ },
+ { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
+ 0x00, 0x00, 0x00, 0x00,
+ },
+ { 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A,
+ 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A,
+ 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A, 0x6D, 0x5A,
+ 0x6D, 0x5A, 0x6D, 0x5A,
+ },
+ { 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0A, 0x0B, 0x0C,
+ 0x0D, 0x0E, 0x0F, 0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17, 0x18,
+ 0x19, 0x1A, 0x1B, 0x1C, 0x1D, 0x1E, 0x1F, 0x20, 0x21, 0x22, 0x23, 0x24,
+ 0x25, 0x26, 0x27, 0x28,
+ }
+};
+
+static void
+win_err_addline(const char *text)
+{
+ int i;
+
+ if (win_err_lines < 5) {
+ win_err_lines++;
+ _do_configure = 1;
+ } else {
+ free(ui_err_text[0]);
+ for (i = 1; i < 5; i++)
+ ui_err_text[i - 1] = ui_err_text[i];
+ }
+
+ ui_err_text[win_err_lines - 1] = strdup(text);
+ _do_repaint_err = 1;
+}
+
+static void
+win_err_fini(void)
+{
+ int i;
+
+ for (i = 0; i < (int)win_err_lines; i++)
+ free(ui_err_text[i]);
+}
+
+/**
+ * Draw full information for port
+ */
+static void
+win_header_draw(uint8_t port_id)
+{
+ int i, j;
+ attr_t a;
+
+ update_port_info(selected_port_id);
+
+ /* Draw title bar */
+ if (ui_mode == UI_MODE_PORT)
+ wattron(win_header, COLOR_PAIR(2));
+ else
+ wattron(win_header, COLOR_PAIR(4));
+
+ mvwprintw(win_header, 0, 0, "::");
+
+ for (i = 0; i < nb_ports; i++) {
+ if (ports[i].enabled) {
+ const char *name = (ports[i].bond_mode >= 0 ? "BOND" : "SLAVE");
+
+ if (i == port_id) {
+ wattron(win_header, A_REVERSE);
+ wprintw(win_header, " %s-%d ", name, (unsigned) i);
+ wattroff(win_header, A_REVERSE);
+ } else
+ wprintw(win_header, " %s-%d ", name, (unsigned) i);
+ }
+ }
+
+ for (i = scr_cols - getcurx(win_header) - 3; i > 0; i--)
+ waddch(win_header, ':');
+
+ waddch(win_header,
+ (ports[port_id].rss ? 'R' : '-'));
+ waddch(win_header, ports[port_id].bond_mode >= 0 ?
+ ports[port_id].bond_mode + '0' : '-');
+ waddch(win_header, ports[port_id].promiscuous == 0 ? '-' : 'P');
+
+ if (ui_mode == UI_MODE_PORT)
+ wattroff(win_header, COLOR_PAIR(2));
+ else
+ wattroff(win_header, COLOR_PAIR(4));
+
+ /* Redraw RSS-Fn */
+ selected_port->rss_type_count = 0;
+ for (i = 0; i < (int) RTE_DIM(rss_type_table); i++) {
+ if (selected_port->dev_info.flow_type_rss_offloads
+ & rss_type_table[i].rss_type) {
+ selected_port->rss_type_fn[selected_port->rss_type_count].info =
+ &rss_type_table[i];
+ selected_port->rss_type_fn[selected_port->rss_type_count].enabled =
+ ((selected_port->dev_conf.rx_adv_conf.rss_conf.rss_hf
+ & rss_type_table[i].rss_type) ? 1 : 0);
+ selected_port->rss_type_count++;
+ }
+ }
+
+ a = (ui_mode == UI_MODE_RSS_FN ? COLOR_PAIR(2) : COLOR_PAIR(1));
+
+ wattron(win_header, a);
+ mvwprintw(win_header, 1, 0, "FN: ");
+ for (i = 0; i < selected_port->rss_type_count; i++) {
+ if (selected_port->rss_type_fn[i].enabled)
+ wattron(win_header, COLOR_PAIR(3));
+ waddstr(win_header, selected_port->rss_type_fn[i].info->str);
+ if (selected_port->rss_type_fn[i].enabled)
+ wattron(win_header, a);
+ waddch(win_header, ' ');
+ }
+ wattroff(win_header, a);
+
+ /* Redraw RSS-Key */
+ if (ui_mode == UI_MODE_RSS_KEY)
+ wattron(win_header, COLOR_PAIR(2));
+ mvwprintw(win_header, 2, 0, "KEY: ");
+ for (i = 0; i < 40; i++)
+ wprintw(win_header, "%02X",
+ selected_port->dev_conf.rx_adv_conf.rss_conf.rss_key[i]);
+ if (ui_mode == UI_MODE_RSS_KEY)
+ wattroff(win_header, COLOR_PAIR(2));
+
+
+ /* Redraw RETA window */
+ int idx, shift;
+
+ if (ui_mode == UI_MODE_RETA)
+ wattron(win_reta, COLOR_PAIR(2));
+
+ for (j = 0; j < selected_port->dev_info.reta_size / 16; j++) {
+ wmove(win_reta, j, 0);
+ for (i = 0; i < 16; i++) {
+ idx = (j*16 + i) / RTE_RETA_GROUP_SIZE;
+ shift = (j*16 + i) % RTE_RETA_GROUP_SIZE;
+ waddch(win_reta, ACS_VLINE);
+ wprintw(win_reta, "%d", selected_port->reta_conf[idx].reta[shift]);
+ }
+ waddch(win_reta, ACS_VLINE);
+ }
+
+ if (ui_mode == UI_MODE_RETA)
+ wattroff(win_reta, COLOR_PAIR(2));
+ wnoutrefresh(win_reta);
+
+
+ /* Stats repaint */
+ if (_do_repaint_stats) {
+ uint64_t total;
+
+ rte_eth_stats_get(selected_port_id, &(selected_port->stats));
+
+ wmove(win_stats, 0, 0);
+ total = 0;
+ for (i = 0; i < selected_port->nb_rx_queues; i++) {
+ wprintw(win_stats, "Queue %d: %10"PRIu64
+ " (%10" PRIu64 ")\n", i, selected_port->rxq_ipackets[i],
+ selected_port->stats.q_ipackets[i]);
+ total += selected_port->rxq_ipackets[i];
+ }
+
+ wprintw(win_stats, " Total: %10"PRIu64
+ " (%10" PRIu64 ")\n", total, selected_port->stats.ipackets);
+
+ _do_repaint_stats = 0;
+ wnoutrefresh(win_stats);
+ }
+
+ if (_do_repaint_err && win_err_lines > 0) {
+ for (i = 0; i < (int)win_err_lines; i++) {
+ mvwprintw(win_err, i, 0, ui_err_text[i]);
+ wclrtoeol(win_err);
+ }
+ _do_repaint_err = 0;
+ wnoutrefresh(win_err);
+ }
+
+ mvwhline(win_header, win_header_lines - 1, 0, 0, scr_cols);
+}
+
+static void
+win_footer_draw(const char *text)
+{
+ if (text != NULL)
+ ui_status_text = text;
+
+ wclear(win_footer);
+ wmove(win_footer, 0, 0);
+ wprintw(win_footer, ui_status_text);
+
+ wprintw(win_footer, "; [arrows]navigate; [q]quit");
+ wnoutrefresh(win_footer);
+}
+
+static void
+win_list_draw(void)
+{
+ unsigned n, i;
+
+ struct pkt_info *pkt_info;
+
+ struct rte_port *port = &ports[focused_port_id];
+ struct pkt_record *record = &(port->record);
+
+ n = (win_list_lines > record->count) ? record->count : win_list_lines;
+
+ wmove(win_list, 0, 0);
+
+ for (i = 0; i < n; i++) {
+
+ pkt_info = &record->pkt_info[(record->top - n + i + PKT_RECORD_SIZE)
+ % PKT_RECORD_SIZE];
+
+ wmove(win_list, i, 0);
+
+ wprintw(win_list, "%5"PRIu64, (unsigned) pkt_info->n);
+ waddch(win_list, ACS_VLINE);
+
+ wprintw(win_list, "%2d: ", (unsigned) pkt_info->queue_id);
+ wprintw(win_list, "%08x ", (unsigned) pkt_info->rss);
+
+ if (pkt_info->ol_flags != 0) {
+ unsigned rxf;
+ const char *name;
+
+ for (rxf = 0; rxf < sizeof(pkt_info->ol_flags) * 8; rxf++) {
+ if ((pkt_info->ol_flags & (1ULL << rxf)) == 0)
+ continue;
+ name = rte_get_rx_ol_flag_name(1ULL << rxf);
+ if (name == NULL)
+ continue;
+ wprintw(win_list, "%s ", name);
+ }
+ }
+ wclrtoeol(win_list);
+ }
+
+ wclrtobot(win_list);
+ wnoutrefresh(win_list);
+}
+
+static void
+ui_repaint(void)
+{
+ switch (ui_mode) {
+ default:
+ win_header_draw(selected_port_id);
+ wnoutrefresh(win_header);
+
+ if (_do_repaint_list) {
+ win_list_draw();
+ _do_repaint_list = 0;
+ }
+
+ break;
+ }
+
+ win_footer_draw(NULL);
+ _do_repaint = 0;
+}
+
+static void
+ui_mode_set(int mode)
+{
+ ui_mode = (mode + 4) % 4;
+ switch (ui_mode) {
+ case UI_MODE_PORT:
+ ui_status_text = "Port: [p]promiscuous; [c]clear stats";
+ break;
+ case UI_MODE_RSS_FN:
+ ui_status_text = "RSS Fn: [1-9]toggle fn";
+ break;
+ case UI_MODE_RETA:
+ ui_status_text =
+ "RETA: [d]set default; [r]randomize; [0-9]fill table with value";
+ break;
+ case UI_MODE_RSS_KEY:
+ ui_status_text = "RSS Key: [r]randomize; [1-4]select predefined key";
+ break;
+ }
+ _do_repaint = 1;
+}
+
+static void
+ui_configure(void)
+{
+ int win_reta_cols = (16 * 2 + 1);
+
+ win_reta_lines = selected_port->dev_info.reta_size / 16;
+ win_stats_lines = selected_port->nb_rx_queues + 1;
+ win_header_lines = win_reta_lines;
+ if (win_header_lines < win_stats_lines)
+ win_header_lines = win_stats_lines;
+ win_header_lines += 5;
+ win_list_lines = scr_lines - win_header_lines - win_err_lines - 1;
+
+ if (win_footer == NULL) {
+ win_footer = newwin(1, scr_cols, scr_lines - 1, 0);
+ ui_status_text = "";
+ wbkgdset(win_footer, COLOR_PAIR(1));
+ } else {
+ wresize(win_footer, 1, scr_cols);
+ mvwin(win_footer, scr_lines - 1, 0);
+ }
+
+ if (win_err == NULL) {
+ win_err = newwin(1, scr_cols, scr_lines - 1 - win_err_lines, 0);
+ ui_status_text = "";
+ wbkgdset(win_err, COLOR_PAIR(5));
+ } else {
+ wresize(win_err, win_err_lines, scr_cols);
+ mvwin(win_err, scr_lines - win_err_lines - 1, 0);
+ }
+
+
+ if (win_header == NULL) {
+ win_header = newwin(win_header_lines, scr_cols, 0, 0);
+ wbkgdset(win_header, COLOR_PAIR(1));
+ } else
+ wresize(win_header, win_header_lines, scr_cols);
+
+ if (win_stats == NULL)
+ win_stats = subwin(win_header, win_stats_lines,
+ scr_cols - win_reta_cols - 7, 4, win_reta_cols + 7);
+ else
+ wresize(win_stats, win_stats_lines, scr_cols - win_reta_cols - 7);
+
+ if (win_reta == NULL)
+ win_reta = subwin(win_header, win_reta_lines, win_reta_cols,
+ 4, 5);
+ else
+ wresize(win_reta, win_reta_lines, win_reta_cols);
+
+ if (win_list == NULL)
+ win_list = newwin(win_list_lines, scr_cols, win_header_lines, 0);
+ else {
+ wresize(win_list, win_list_lines, scr_cols);
+ mvwin(win_list, win_header_lines, 0);
+ }
+
+ wclear(win_header);
+ wclear(win_reta);
+
+ _do_configure = 0;
+ _do_repaint = 1;
+ _do_repaint_list = 1;
+ _do_repaint_stats = 1;
+ _do_repaint_info = 1;
+ _do_repaint_err = 1;
+
+}
+
+static void
+ui_resize(void)
+{
+ if (COLS != scr_cols || LINES != scr_lines) {
+ scr_cols = COLS;
+ scr_lines = LINES;
+ _do_configure = 1;
+ }
+}
+
+static void
+ui_update(void)
+{
+ if (ui_lock(1)) {
+ if (_is_ready) {
+ if (_do_configure)
+ ui_configure();
+ if (_do_repaint) {
+ ui_repaint();
+ doupdate();
+ }
+ }
+ ui_unlock();
+ }
+}
+
+static void
+ui_fini(void)
+{
+ win_err_fini();
+ endwin();
+}
+
+static void
+ui_init(void)
+{
+ initscr();
+ start_color();
+ init_pair(1, COLOR_YELLOW, COLOR_BLUE);
+ init_pair(2, COLOR_BLACK, COLOR_CYAN);
+ init_pair(3, COLOR_BLACK, COLOR_YELLOW);
+ init_pair(4, COLOR_BLACK, COLOR_WHITE);
+ init_pair(5, COLOR_YELLOW, COLOR_RED);
+
+ cbreak();
+ noecho();
+ curs_set(FALSE);
+ keypad(stdscr, TRUE);
+ timeout(100);
+
+ scr_cols = COLS;
+ scr_lines = LINES;
+ _do_configure = 1;
+ _is_ready = 1;
+}
+
+static int
+port_select(uint8_t port_id)
+{
+ if (ports[port_id].enabled == 1) {
+ focused_port_id = selected_port_id = port_id;
+
+ selected_port = &ports[selected_port_id];
+ focused_port = &ports[focused_port_id];
+
+ update_port_info(selected_port_id);
+
+ _do_configure = 1;
+ _do_repaint = 1;
+ _do_repaint_info = 1;
+ _do_repaint_list = 1;
+ _do_repaint_stats = 1;
+ return selected_port_id;
+ }
+ return -1;
+}
+
+static void
+port_select_prev(void)
+{
+ int i;
+
+ for (i = 1; i < nb_ports; i++)
+ if (port_select((selected_port_id - i + nb_ports) % nb_ports) != -1)
+ break;
+}
+
+static void
+port_select_next(void)
+{
+ int i;
+
+ for (i = 1; i < nb_ports; i++)
+ if (port_select((selected_port_id + i) % nb_ports) != -1)
+ break;
+}
+
+/**
+ * Toggle on/off RSS fn for focused port
+ */
+static void
+rss_fn_toggle(int index)
+{
+ int i;
+ uint64_t rss_type = 0;
+
+ if (index < 0 || index >= focused_port->rss_type_count)
+ return;
+
+ focused_port->rss_type_fn[index].enabled ^= 1; /* toggle */
+
+ for (i = 0; i < focused_port->rss_type_count; i++)
+ if (focused_port->rss_type_fn[i].enabled)
+ rss_type |= focused_port->rss_type_fn[i].info->rss_type;
+
+ focused_port->dev_conf.rx_adv_conf.rss_conf.rss_hf = rss_type;
+
+ rte_eth_dev_rss_hash_update(focused_port_id,
+ &focused_port->dev_conf.rx_adv_conf.rss_conf);
+}
+
+/**
+ * Set RSS key of focused port from predefined table of keys
+ */
+static int
+key_set_predefined(int index)
+{
+ if (index < 0 || index >= (int) RTE_DIM(RSS_KEY))
+ return -1;
+
+ return key_set(focused_port_id, RSS_KEY[index], 40);
+}
+
+/**
+ * Record packet from mbuf
+ */
+struct pkt_info *
+record_pkt(uint8_t port_id, int queue_id, struct rte_mbuf *mb)
+{
+ struct rte_port *port = &ports[port_id];
+ struct pkt_record *record = &(port->record);
+ struct pkt_info *pkt_info = &(record->pkt_info[record->top]);
+
+ ui_lock(1);
+
+ record->count++;
+ pkt_info->n = mb->udata64;
+ pkt_info->ol_flags = mb->ol_flags;
+ pkt_info->queue_id = queue_id;
+ pkt_info->rss = mb->hash.rss;
+
+ record->top = (record->top + 1) % PKT_RECORD_SIZE;
+
+ _do_repaint = 1;
+ _do_repaint_list = 1;
+ _do_repaint_stats = 1;
+ ui_unlock();
+
+ return pkt_info;
+}
+
+static void
+ui_help(void) {
+ timeout(-1);
+
+ WINDOW *win_help_border = NULL;
+ WINDOW *win_help = NULL;
+ char title[256];
+ int page = 0;
+ int page_prev;
+ int page_next;
+ int c;
+ int cols;
+ int lines;
+ int top;
+ int left;
+
+ endwin();
+ initscr();
+ ui_resize();
+ ui_update();
+
+ cols = scr_cols > 80 ? 80 : scr_cols;
+ lines = scr_lines > 25 ? 25 : scr_lines;
+ top = (scr_lines - lines) / 2;
+ left = (scr_cols - cols) / 2;
+
+ win_help_border = newwin(lines, cols, top, left);
+ win_help = derwin(win_help_border, lines - 2, cols - 4, 1, 2);
+ wbkgdset(win_help_border, COLOR_PAIR(2));
+ wbkgdset(win_help, COLOR_PAIR(2));
+
+ do {
+ page_prev = (page + RTE_DIM(help_pages) - 1) % RTE_DIM(help_pages);
+ page_next = (page + 1) % RTE_DIM(help_pages);
+
+ wclear(win_help_border);
+ box(win_help_border, 0, 0);
+
+ sprintf(title, "[ Help: %s ]", help_pages[page].title);
+ mvwprintw(win_help_border, 0, (cols - strlen(title)) / 2 - 2, "%s",
+ title);
+
+ sprintf(title, "< %s ]", help_pages[page_prev].title);
+ mvwprintw(win_help_border, lines - 1, 1, "%s", title);
+
+ sprintf(title, "[ Page %d of %d ]", page + 1, (int)RTE_DIM(help_pages));
+ mvwprintw(win_help_border, lines - 1, (cols - strlen(title)) / 2 - 2,
+ "%s", title);
+
+ sprintf(title, "[ %s >", help_pages[page_next].title);
+ mvwprintw(win_help_border, lines - 1, cols - strlen(title) - 1, "%s",
+ title);
+
+ wrefresh(win_help_border);
+
+ wclear(win_help);
+
+ wmove(win_help, 0, 0);
+
+ wprintw(win_help, help_pages[page].body);
+
+ wrefresh(win_help);
+
+ switch (c = getch()) {
+
+ case KEY_RESIZE:
+ endwin();
+ initscr();
+ ui_resize();
+ ui_update();
+
+ cols = scr_cols > 80 ? 80 : scr_cols;
+ lines = scr_lines > 25 ? 25 : scr_cols;
+ top = (scr_lines - lines) / 2;
+ left = (scr_cols - cols) / 2;
+
+ wresize(win_help_border, lines, cols);
+ wresize(win_help, lines - 2, cols - 4);
+ break;
+
+ case KEY_LEFT:
+ page = page_prev;
+ break;
+
+ case KEY_RIGHT:
+ page = page_next;
+ break;
+
+ default:
+ c = -1; /* Exit */
+ break;
+ }
+
+
+ } while (c != -1);
+
+ timeout(100);
+ delwin(win_help);
+ delwin(win_help_border);
+ _do_configure = 1;
+ _do_repaint = 1;
+}
+
+/* main processing loop */
+void
+ui_loop(void)
+{
+ int c;
+ int i;
+ int port_id;
+ int refresh = 0;
+
+ ui_init();
+ port_select(bond_port_id);
+
+ ui_mode_set(UI_MODE_PORT);
+
+ while ((c = getch()) != 'q') {
+ switch (c) {
+ case -1:
+ refresh++;
+ if (refresh % 10)
+ _do_configure = 1;
+ ui_update();
+ break;
+
+ case 'c':
+ /* clear stats */
+ ui_lock(1);
+
+ rte_atomic64_clear(&counter);
+ for (port_id = 0; port_id < RTE_MAX_ETHPORTS; port_id++) {
+ if (ports[port_id].enabled) {
+ ports[port_id].record.count = 0;
+ ports[port_id].record.top = 0;
+ for (i = 0; i < ports[port_id].nb_rx_queues; i++)
+ ports[port_id].rxq_ipackets[i] = 0;
+ rte_eth_stats_reset(port_id);
+ }
+ }
+
+ _do_repaint = 1;
+ ui_unlock();
+ break;
+
+ case ' ':
+ focused_port->rss ^= 1;
+ rss_enable(focused_port_id, focused_port->rss);
+ _do_configure = 1;
+ _do_repaint = 1;
+ break;
+
+ case 'p':
+ if (focused_port->promiscuous)
+ rte_eth_promiscuous_disable(focused_port_id);
+ else
+ rte_eth_promiscuous_enable(focused_port_id);
+ focused_port->promiscuous ^= 1;
+ _do_configure = 1;
+ _do_repaint = 1;
+ break;
+
+ case KEY_RESIZE:
+ ui_resize();
+ break;
+
+ case KEY_UP:
+ ui_mode_set(ui_mode - 1);
+ break;
+ case KEY_DOWN:
+ ui_mode_set(ui_mode + 1);
+ break;
+ case 'h':
+ case '?':
+ ui_help();
+ break;
+
+ default:
+ switch (ui_mode) {
+ case UI_MODE_PORT:
+ switch (c) {
+ case KEY_LEFT:
+ port_select_prev();
+ break;
+
+ case KEY_RIGHT:
+ port_select_next();
+ break;
+
+ default:
+ i = (c - '1');
+ port_select(i);
+ break;
+ }
+ break;
+
+ case UI_MODE_RSS_KEY:
+ switch (c) {
+ case 'r':
+ if (key_set_random(focused_port_id, 40) < 0)
+ win_err_addline("Cannot update RSS key");
+ break;
+
+ default:
+ c = c - '1';
+ if (key_set_predefined(c) < 0)
+ win_err_addline("Cannot update RSS key");
+ break;
+ }
+ update_port_info(focused_port_id);
+ break;
+
+ case UI_MODE_RETA:
+ switch (c) {
+ case 'r':
+ reta_set_random(focused_port_id);
+ break;
+ case 'd':
+ reta_set_default(focused_port_id);
+ break;
+ default:
+ c = c - '0';
+ reta_set_all(focused_port_id, c);
+ break;
+ }
+ break;
+
+ case UI_MODE_RSS_FN:
+ switch (c) {
+ case 'a':
+ /* Set all */
+ focused_port->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ ports[focused_port_id].dev_info.flow_type_rss_offloads;
+ rte_eth_dev_rss_hash_update(focused_port_id,
+ &focused_port->dev_conf.rx_adv_conf.rss_conf);
+ break;
+
+ default:
+ c -= '1';
+ rss_fn_toggle(c);
+ }
+ break;
+
+ }
+ _do_repaint = 1;
+ break;
+ }
+ }
+ ui_fini();
+}
--
1.7.9.5
Thomas Monjalon
2015-07-13 11:20:22 UTC
Permalink
Post by Tomasz Kulasek
This application allows you to test RSS configuration for bonded devices
changing configuration dynamically, during receiving packets on slaves and
observe the changes in its distribution over queues.
After initialization process, all accessible ports are attached to one
bonding as slaves.
- Port selection (on the very top)
- RSS Configuration for selected port including hash function, key and
RETA
- Incoming packets statistics
- Incoming packets list for selected port
- Status bar with contextual information about selected part
---
examples/bond_rss/Makefile | 59 +++
examples/bond_rss/bondrss.c | 293 ++++++++++++++
examples/bond_rss/bondrss.h | 163 ++++++++
examples/bond_rss/config.c | 251 ++++++++++++
examples/bond_rss/ui.c | 945 +++++++++++++++++++++++++++++++++++++++++++
5 files changed, 1711 insertions(+)
This new example is not added in examples/Makefile.
It introduces a new dependency (ncurses).

Examples are useful to show how to use some features. Maybe you can show
bonding RSS in the existing bonding example without adding bells and whistles.
The example directory must be kept reasonnably maintainable.

REJECTED
Tomasz Kulasek
2015-06-29 14:50:43 UTC
Permalink
Signed-off-by: Tomasz Kulasek <***@intel.com>
---
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
index 96e554f..03baf90 100644
--- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
+++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
@@ -188,7 +188,7 @@ conditions are not met. If a user wishes to monitor individual slaves then they
must register callbacks with that slave directly.

The link bonding library also supports devices which do not implement link
-status change interrupts, this is achieve by polling the devices link status at
+status change interrupts, this is achieved by polling the devices link status at
a defined period which is set using the ``rte_eth_bond_link_monitoring_set``
API, the default polling interval is 10ms. When a device is added as a slave to
a bonding device it is determined using the ``RTE_PCI_DRV_INTR_LSC`` flag
@@ -286,7 +286,7 @@ and UDP protocols for load balancing.
Using Link Bonding Devices
--------------------------

-The librte_pmd_bond library support two modes of device creation, the libraries
+The librte_pmd_bond library supports two modes of device creation, the libraries
export full C API or using the EAL command line to statically configure link
bonding devices at application startup. Using the EAL option it is possible to
use link bonding functionality transparently without specific knowledge of the
@@ -299,7 +299,7 @@ Using the Poll Mode Driver from an Application

Using the librte_pmd_bond libraries API it is possible to dynamically create
and manage link bonding device from within any application. Link bonding
-device are created using the ``rte_eth_bond_create`` API which requires a
+devices are created using the ``rte_eth_bond_create`` API which requires a
unique device name, the link bonding mode to initial the device in and finally
the socket Id which to allocate the devices resources onto. After successful
creation of a bonding device it must be configured using the generic Ethernet
@@ -362,7 +362,7 @@ The different options are:
mode=2

* slave: Defines the PMD device which will be added as slave to the bonded
- device. This option can be selected multiple time, for each device to be
+ device. This option can be selected multiple times, for each device to be
added as a slave. Physical devices should be specified using their PCI
address, in the format domain:bus:devid.function
--
1.7.9.5
Tomasz Kulasek
2015-06-29 14:50:44 UTC
Permalink
Documentation update about implementation details and requirements for
Dynamic RSS Configuration for Bonding.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 34 ++++++++++++++++++--
1 file changed, 32 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
index 03baf90..46f0296 100644
--- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
+++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
@@ -1,5 +1,5 @@
.. BSD LICENSE
- Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+ Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
All rights reserved.

Redistribution and use in source and binary forms, with or without
@@ -173,7 +173,28 @@ After a slave device is added to a bonded device slave is stopped using
``rte_eth_dev_stop`` and then reconfigured using ``rte_eth_dev_configure``
the RX and TX queues are also reconfigured using ``rte_eth_tx_queue_setup`` /
``rte_eth_rx_queue_setup`` with the parameters use to configure the bonding
-device.
+device. If RSS is enabled for bonding device, this mode is also enabled on new
+slave and configured as well.
+
+Setting up multi-queue mode for bonding device to RSS, makes it fully
+RSS-capable, so all slaves are synchronized with its configuration. This mode is
+intended to provide RSS configuration on slaves transparent for client
+application implementation.
+
+Bonding device stores its own version of RSS settings i.e. RETA, RSS hash
+function and RSS key, used to set up its slaves. That let to define the meaning
+of RSS configuration of bonding device as desired configuration of whole bonding
+(as one unit), without pointing any of slave inside. It is required to ensure
+consistency and made it more errorproof.
+
+RSS hash function set for bonding device, is a maximal set of RSS hash functions
+supported by all bonded slaves. RETA size is a GCD of all its RETA's sizes, so
+it can be easily used as a pattern providing expected behavior, even if slave
+RETAs' sizes are different. If RSS Key is not set for bonded device, it's not
+changed on the slaves and default key for device is used.
+
+All settings are managed through the bonding port API and always are propagated
+in one direction (from bonding to slaves).

Link Status Change Interrupts / Polling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -207,6 +228,15 @@ these parameters.
A bonding device must have a minimum of one slave before the bonding device
itself can be started.

+To use a bonding device dynamic RSS configuration feature effectively, it is
+also required, that all slaves should be RSS-capable and support, at least one
+common hash function available for each of them. Changing RSS key is only
+possible, when all slave devices support the same key size.
+
+To prevent inconsistency on how slaves process packets, once a device is added
+to a bonding device, RSS configuration should be managed through the bonding
+device API, and not directly on the slave.
+
Like all other PMD, all functions exported by a PMD are lock-free functions
that are assumed not to be invoked in parallel on different logical cores to
work on the same target object.
--
1.7.9.5
Declan Doherty
2015-07-01 10:05:07 UTC
Permalink
Post by Tomasz Kulasek
OVERVIEW
--------
1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding
device fully RSS-capable, so all slaves are synchronized with its configuration.
This mode is intended to provide RSS configuration as known from "dynamic RSS
configuration for one port" and made slaves transparent for client application
implementation.
2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not
synchronized. That provides an ability to configure them manually. This mode may
be useful when application wants to manage RSS in an unusual way and the
consistency of RSS configuration for slaves isn't required.
Turning on/off RSS mode for slaves when bonding is started is not possible.
Other RSS configuration is propagated over slaves, when bonding device API is
used to do it.
- checkpatch cleanups
...
Acked-by : Declan Doherty <***@intel.com>
Thomas Monjalon
2015-07-13 11:03:21 UTC
Permalink
Post by Tomasz Kulasek
OVERVIEW
--------
1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding
device fully RSS-capable, so all slaves are synchronized with its configuration.
This mode is intended to provide RSS configuration as known from "dynamic RSS
configuration for one port" and made slaves transparent for client application
implementation.
2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not
synchronized. That provides an ability to configure them manually. This mode may
be useful when application wants to manage RSS in an unusual way and the
consistency of RSS configuration for slaves isn't required.
Turning on/off RSS mode for slaves when bonding is started is not possible.
Other RSS configuration is propagated over slaves, when bonding device API is
used to do it.
- checkpatch cleanups
...
Applied without patches 5 and 6:

- As discussed earlier, patch 5 workaround a missing flag to announce stats
per queue availability.

- Patch 6 introduce a new dependency (ncurses) to build a new bond_rss example.
Examples are useful to show how to use some features. Maybe you can show
bonding RSS in the existing bonding example without adding bells and whistles.
The example directory must be kept reasonnably maintainable.
Thomas Monjalon
2015-07-13 11:18:13 UTC
Permalink
Post by Thomas Monjalon
Post by Tomasz Kulasek
OVERVIEW
--------
1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding
device fully RSS-capable, so all slaves are synchronized with its configuration.
This mode is intended to provide RSS configuration as known from "dynamic RSS
configuration for one port" and made slaves transparent for client application
implementation.
2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not
synchronized. That provides an ability to configure them manually. This mode may
be useful when application wants to manage RSS in an unusual way and the
consistency of RSS configuration for slaves isn't required.
Turning on/off RSS mode for slaves when bonding is started is not possible.
Other RSS configuration is propagated over slaves, when bonding device API is
used to do it.
- checkpatch cleanups
...
Sorry, after more review, this series won't be pushed at all.
There is an obvious error in patch 4, and patch 2/3 must be discussed.
Post by Thomas Monjalon
- As discussed earlier, patch 5 workaround a missing flag to announce stats
per queue availability.
- Patch 6 introduce a new dependency (ncurses) to build a new bond_rss example.
Examples are useful to show how to use some features. Maybe you can show
bonding RSS in the existing bonding example without adding bells and whistles.
The example directory must be kept reasonnably maintainable.
Kulasek, TomaszX
2015-07-13 12:14:41 UTC
Permalink
-----Original Message-----
Sent: Monday, July 13, 2015 13:18
To: Kulasek, TomaszX
Subject: Re: [dpdk-dev] [PATCH v3 0/8] Dynamic RSS Configuration for
Bonding
Post by Thomas Monjalon
Post by Tomasz Kulasek
OVERVIEW
--------
1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS
makes bonding device fully RSS-capable, so all slaves are
synchronized with its configuration.
Post by Thomas Monjalon
Post by Tomasz Kulasek
This mode is intended to provide RSS configuration as known from
"dynamic RSS configuration for one port" and made slaves
transparent for client application implementation.
2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS,
slaves are not synchronized. That provides an ability to configure
them manually. This mode may be useful when application wants to
manage RSS in an unusual way and the consistency of RSS
configuration for slaves isn't required.
Post by Thomas Monjalon
Post by Tomasz Kulasek
Turning on/off RSS mode for slaves when bonding is started is not
possible.
Post by Thomas Monjalon
Post by Tomasz Kulasek
Other RSS configuration is propagated over slaves, when bonding
device API is used to do it.
- checkpatch cleanups
...
Sorry, after more review, this series won't be pushed at all.
There is an obvious error in patch 4, and patch 2/3 must be discussed.
What's wrong with patches 2/3? Some details?
Post by Thomas Monjalon
- As discussed earlier, patch 5 workaround a missing flag to announce
stats per queue availability.
- Patch 6 introduce a new dependency (ncurses) to build a new bond_rss
example.
Post by Thomas Monjalon
Examples are useful to show how to use some features. Maybe you can
show bonding RSS in the existing bonding example without adding bells
and whistles.
Post by Thomas Monjalon
The example directory must be kept reasonnably maintainable.
Ok. I understand that. This application was created to help me in development process and using ncurses made it a lot easier. The intention wasn't "adding bells and whistles" but made it usable in dynamic environment.
Tomasz Kulasek
2015-07-15 17:26:20 UTC
Permalink
OVERVIEW
--------
1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding
device fully RSS-capable, so all slaves are synchronized with its configuration.
This mode is intended to provide RSS configuration as known from "dynamic RSS
configuration for one port" and made slaves transparent for client application
implementation.

2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not
synchronized. That provides an ability to configure them manually. This mode may
be useful when application wants to manage RSS in an unusual way and the
consistency of RSS configuration for slaves isn't required.

Turning on/off RSS mode for slaves when bonding is started is not possible.
Other RSS configuration is propagated over slaves, when bonding device API is
used to do it.

v4 changes:
- fixed copy-paste error,
- removed example application as too complex and introducing a new
dependency,
- addapted null pmd to be used as testing device for dynamic RSS configuration,
- addapted test units to use null pmd instead of ring pmd,
- ring pmd is not used and changed in this patchset,

Tomasz Kulasek (9):
bonding: rss dynamic configuration
null: fix segfault when null_pmd added to bonding
null: extend number of virtual queues
null: virtual dynamic rss configuration
null: export eth_dev_null_create
test: dynamic rss configuration
bonding: queue stats mapping
doc: fixed spellings and typos
doc: dynamic rss configuration for bonding

app/test/Makefile | 8 +
app/test/test_link_bonding_rssconf.c | 679 ++++++++++++++++++++
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 42 +-
drivers/net/bonding/rte_eth_bond_api.c | 28 +
drivers/net/bonding/rte_eth_bond_pmd.c | 239 ++++++-
drivers/net/bonding/rte_eth_bond_private.h | 12 +
drivers/net/null/Makefile | 2 +-
drivers/net/null/rte_eth_null.c | 142 +++-
drivers/net/null/rte_eth_null.h | 40 ++
drivers/net/null/rte_pmd_null_version.map | 7 +
10 files changed, 1170 insertions(+), 29 deletions(-)
create mode 100644 app/test/test_link_bonding_rssconf.c
create mode 100644 drivers/net/null/rte_eth_null.h
--
1.7.9.5
Tomasz Kulasek
2015-07-15 17:26:21 UTC
Permalink
Bonding device implements independent management of RSS settings. It
stores its own copies of settings i.e. RETA, RSS hash function and RSS
key. It’s required to ensure consistency.

1) RSS hash function set for bonding device is maximal set of RSS hash
functions supported by all bonded devices. That mean, to have RSS support
for bonding, all slaves should be RSS-capable.

2) RSS key is propagated over the slaves "as is".

3) RETA for bonding is an internal table managed by bonding API, and is
used as a pattern to set up slaves. Its size is GCD of all RETA sizes, so
it can be easily used as a pattern providing expected behavior, even if
slaves RETA sizes are different.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/bonding/rte_eth_bond_api.c | 28 ++++
drivers/net/bonding/rte_eth_bond_pmd.c | 205 ++++++++++++++++++++++++++--
drivers/net/bonding/rte_eth_bond_private.h | 12 ++
3 files changed, 231 insertions(+), 14 deletions(-)

diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c
index d810ec4..22eb575 100644
--- a/drivers/net/bonding/rte_eth_bond_api.c
+++ b/drivers/net/bonding/rte_eth_bond_api.c
@@ -303,6 +303,9 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id)
internals->rx_offload_capa = 0;
internals->tx_offload_capa = 0;

+ /* Initially allow to choose any offload type */
+ internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK;
+
memset(internals->active_slaves, 0, sizeof(internals->active_slaves));
memset(internals->slaves, 0, sizeof(internals->slaves));

@@ -366,6 +369,11 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)

rte_eth_dev_info_get(slave_port_id, &dev_info);

+ /* We need to store slaves reta_size to be able to synchronize RETA for all
+ * slave devices even if its sizes are different.
+ */
+ internals->slaves[internals->slave_count].reta_size = dev_info.reta_size;
+
if (internals->slave_count < 1) {
/* if MAC is not user defined then use MAC of first slave add to
* bonded device */
@@ -379,9 +387,16 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
/* Make primary slave */
internals->primary_port = slave_port_id;

+ /* Inherit queues settings from first slave */
+ internals->nb_rx_queues = slave_eth_dev->data->nb_rx_queues;
+ internals->nb_tx_queues = slave_eth_dev->data->nb_tx_queues;
+
+ internals->reta_size = dev_info.reta_size;
+
/* Take the first dev's offload capabilities */
internals->rx_offload_capa = dev_info.rx_offload_capa;
internals->tx_offload_capa = dev_info.tx_offload_capa;
+ internals->flow_type_rss_offloads = dev_info.flow_type_rss_offloads;

} else {
/* Check slave link properties are supported if props are set,
@@ -400,8 +415,19 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
}
internals->rx_offload_capa &= dev_info.rx_offload_capa;
internals->tx_offload_capa &= dev_info.tx_offload_capa;
+ internals->flow_type_rss_offloads &= dev_info.flow_type_rss_offloads;
+
+ /* RETA size is GCD of all slaves RETA sizes, so, if all sizes will be
+ * the power of 2, the lower one is GCD
+ */
+ if (internals->reta_size > dev_info.reta_size)
+ internals->reta_size = dev_info.reta_size;
+
}

+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf &=
+ internals->flow_type_rss_offloads;
+
internals->slave_count++;

/* Update all slave devices MACs*/
@@ -528,6 +554,8 @@ __eth_bond_slave_remove_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
if (internals->slave_count == 0) {
internals->rx_offload_capa = 0;
internals->tx_offload_capa = 0;
+ internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK;
+ internals->reta_size = 0;
}
return 0;
}
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 989e878..cd23f42 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -1310,6 +1310,23 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev,
if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC)
slave_eth_dev->data->dev_conf.intr_conf.lsc = 1;

+ /* If RSS is enabled for bonding, try to enable it for slaves */
+ if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) {
+ if (bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len
+ != 0) {
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len =
+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len;
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key =
+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key;
+ } else {
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL;
+ }
+
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ slave_eth_dev->data->dev_conf.rxmode.mq_mode |= ETH_MQ_RX_RSS;
+ }
+
/* Configure device */
errval = rte_eth_dev_configure(slave_eth_dev->data->port_id,
bonded_eth_dev->data->nb_rx_queues,
@@ -1361,6 +1378,30 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev,
return -1;
}

+ /* If RSS is enabled for bonding, synchronize RETA */
+ if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) {
+ int i;
+ struct bond_dev_private *internals;
+
+ internals = bonded_eth_dev->data->dev_private;
+
+ for (i = 0; i < internals->slave_count; i++) {
+ if (internals->slaves[i].port_id == slave_eth_dev->data->port_id) {
+ errval = rte_eth_dev_rss_reta_update(
+ slave_eth_dev->data->port_id,
+ &internals->reta_conf[0],
+ internals->slaves[i].reta_size);
+ if (errval != 0) {
+ RTE_LOG(WARNING, PMD,
+ "rte_eth_dev_rss_reta_update on slave port %d fails (err %d)."
+ " RSS Configuration for bonding may be inconsistent.\n",
+ slave_eth_dev->data->port_id, errval);
+ }
+ break;
+ }
+ }
+ }
+
/* If lsc interrupt is set, check initial slave's link status */
if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC)
bond_ethdev_lsc_event_callback(slave_eth_dev->data->port_id,
@@ -1578,6 +1619,9 @@ bond_ethdev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)

dev_info->rx_offload_capa = internals->rx_offload_capa;
dev_info->tx_offload_capa = internals->tx_offload_capa;
+ dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads;
+
+ dev_info->reta_size = internals->reta_size;
}

static int
@@ -1959,21 +2003,132 @@ bond_ethdev_lsc_event_callback(uint8_t port_id, enum rte_eth_event_type type,
}
}

+static int
+bond_ethdev_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ unsigned i, j;
+ int result = 0;
+ int slave_reta_size;
+ unsigned reta_count;
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ if (reta_size != internals->reta_size)
+ return -EINVAL;
+
+ /* Copy RETA table */
+ reta_count = reta_size / RTE_RETA_GROUP_SIZE;
+
+ for (i = 0; i < reta_count; i++) {
+ internals->reta_conf[i].mask = reta_conf[i].mask;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ internals->reta_conf[i].reta[j] = reta_conf[i].reta[j];
+ }
+
+ /* Fill rest of array */
+ for (; i < RTE_DIM(internals->reta_conf); i += reta_count)
+ memcpy(&internals->reta_conf[i], &internals->reta_conf[0],
+ sizeof(internals->reta_conf[0]) * reta_count);
+
+ /* Propagate RETA over slaves */
+ for (i = 0; i < internals->slave_count; i++) {
+ slave_reta_size = internals->slaves[i].reta_size;
+ result = rte_eth_dev_rss_reta_update(internals->slaves[i].port_id,
+ &internals->reta_conf[0], slave_reta_size);
+ if (result < 0)
+ return result;
+ }
+
+ return 0;
+}
+
+static int
+bond_ethdev_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ int i, j;
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ if (reta_size != internals->reta_size)
+ return -EINVAL;
+
+ /* Copy RETA table */
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++)
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ reta_conf[i].reta[j] = internals->reta_conf[i].reta[j];
+
+ return 0;
+}
+
+static int
+bond_ethdev_rss_hash_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ int i, result = 0;
+ struct bond_dev_private *internals = dev->data->dev_private;
+ struct rte_eth_rss_conf bond_rss_conf;
+
+ memcpy(&bond_rss_conf, rss_conf, sizeof(struct rte_eth_rss_conf));
+
+ bond_rss_conf.rss_hf &= internals->flow_type_rss_offloads;
+
+ if (bond_rss_conf.rss_hf != 0)
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = bond_rss_conf.rss_hf;
+
+ if (bond_rss_conf.rss_key && bond_rss_conf.rss_key_len <
+ sizeof(internals->rss_key)) {
+ if (bond_rss_conf.rss_key_len == 0)
+ bond_rss_conf.rss_key_len = 40;
+ internals->rss_key_len = bond_rss_conf.rss_key_len;
+ memcpy(internals->rss_key, bond_rss_conf.rss_key,
+ internals->rss_key_len);
+ }
+
+ for (i = 0; i < internals->slave_count; i++) {
+ result = rte_eth_dev_rss_hash_update(internals->slaves[i].port_id,
+ &bond_rss_conf);
+ if (result < 0)
+ return result;
+ }
+
+ return 0;
+}
+
+static int
+bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ rss_conf->rss_key_len = internals->rss_key_len;
+ if (rss_conf->rss_key)
+ memcpy(rss_conf->rss_key, internals->rss_key, internals->rss_key_len);
+
+ return 0;
+}
+
struct eth_dev_ops default_dev_ops = {
- .dev_start = bond_ethdev_start,
- .dev_stop = bond_ethdev_stop,
- .dev_close = bond_ethdev_close,
- .dev_configure = bond_ethdev_configure,
- .dev_infos_get = bond_ethdev_info,
- .rx_queue_setup = bond_ethdev_rx_queue_setup,
- .tx_queue_setup = bond_ethdev_tx_queue_setup,
- .rx_queue_release = bond_ethdev_rx_queue_release,
- .tx_queue_release = bond_ethdev_tx_queue_release,
- .link_update = bond_ethdev_link_update,
- .stats_get = bond_ethdev_stats_get,
- .stats_reset = bond_ethdev_stats_reset,
- .promiscuous_enable = bond_ethdev_promiscuous_enable,
- .promiscuous_disable = bond_ethdev_promiscuous_disable
+ .dev_start = bond_ethdev_start,
+ .dev_stop = bond_ethdev_stop,
+ .dev_close = bond_ethdev_close,
+ .dev_configure = bond_ethdev_configure,
+ .dev_infos_get = bond_ethdev_info,
+ .rx_queue_setup = bond_ethdev_rx_queue_setup,
+ .tx_queue_setup = bond_ethdev_tx_queue_setup,
+ .rx_queue_release = bond_ethdev_rx_queue_release,
+ .tx_queue_release = bond_ethdev_tx_queue_release,
+ .link_update = bond_ethdev_link_update,
+ .stats_get = bond_ethdev_stats_get,
+ .stats_reset = bond_ethdev_stats_reset,
+ .promiscuous_enable = bond_ethdev_promiscuous_enable,
+ .promiscuous_disable = bond_ethdev_promiscuous_disable,
+ .reta_update = bond_ethdev_rss_reta_update,
+ .reta_query = bond_ethdev_rss_reta_query,
+ .rss_hash_update = bond_ethdev_rss_hash_update,
+ .rss_hash_conf_get = bond_ethdev_rss_hash_conf_get
};

static int
@@ -2054,6 +2209,28 @@ bond_ethdev_configure(struct rte_eth_dev *dev)
int arg_count;
uint8_t port_id = dev - rte_eth_devices;

+ static const uint8_t default_rss_key[40] = {
+ 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D,
+ 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+ 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B,
+ 0xBE, 0xAC, 0x01, 0xFA
+ };
+
+ unsigned i, j;
+
+ /* If RSS is enabled, fill table and key with default values */
+ if (dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) {
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = internals->rss_key;
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 0;
+ memcpy(internals->rss_key, default_rss_key, 40);
+
+ for (i = 0; i < RTE_DIM(internals->reta_conf); i++) {
+ internals->reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ internals->reta_conf[i].reta[j] = j % dev->data->nb_rx_queues;
+ }
+ }
+
/*
* if no kvlist, it means that this bonded device has been created
* through the bonding api.
diff --git a/drivers/net/bonding/rte_eth_bond_private.h b/drivers/net/bonding/rte_eth_bond_private.h
index 45e5c65..98bb64d 100644
--- a/drivers/net/bonding/rte_eth_bond_private.h
+++ b/drivers/net/bonding/rte_eth_bond_private.h
@@ -103,6 +103,8 @@ struct bond_slave_details {
uint8_t last_link_status;
/**< Port Id of slave eth_dev */
struct ether_addr persisted_mac_addr;
+
+ uint16_t reta_size;
};


@@ -155,6 +157,16 @@ struct bond_dev_private {
uint32_t rx_offload_capa; /** Rx offload capability */
uint32_t tx_offload_capa; /** Tx offload capability */

+ /** Bit mask of RSS offloads, the bit offset also means flow type */
+ uint64_t flow_type_rss_offloads;
+
+ uint16_t reta_size;
+ struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_512 /
+ RTE_RETA_GROUP_SIZE];
+
+ uint8_t rss_key[52]; /**< 52-byte hash key buffer. */
+ uint8_t rss_key_len; /**< hash key length in bytes. */
+
struct rte_kvargs *kvlist;
uint8_t slave_update_idx;
};
--
1.7.9.5
Tomasz Kulasek
2015-07-15 17:26:22 UTC
Permalink
When device is added to the bonding, the link status callback is added to
the slave's eth_dev->link_intr_cbs list. This list is not initialized for
null pmd and adding it to the bonding segfaults application.

This patch allocates and sets up required structures.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/null/rte_eth_null.c | 11 +++++++++++
1 file changed, 11 insertions(+)

diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index e244595..a8b3191 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -386,6 +386,7 @@ eth_dev_null_create(const char *name,
const unsigned nb_rx_queues = 1;
const unsigned nb_tx_queues = 1;
struct rte_eth_dev_data *data = NULL;
+ struct eth_driver *eth_drv = NULL;
struct rte_pci_device *pci_dev = NULL;
struct pmd_internals *internals = NULL;
struct rte_eth_dev *eth_dev = NULL;
@@ -416,6 +417,10 @@ eth_dev_null_create(const char *name,
if (eth_dev == NULL)
goto error;

+ eth_drv = rte_zmalloc_socket(name, sizeof(*eth_drv), 0, numa_node);
+ if (!eth_drv)
+ goto error;
+
/* now put it all together
* - store queue data in internals,
* - store numa_node info in pci_driver
@@ -431,7 +436,10 @@ eth_dev_null_create(const char *name,
internals->packet_copy = packet_copy;
internals->numa_node = numa_node;

+ eth_drv->pci_drv.name = drivername;
+
pci_dev->numa_node = numa_node;
+ pci_dev->driver = &eth_drv->pci_drv;

data->dev_private = internals;
data->port_id = eth_dev->data->port_id;
@@ -445,6 +453,7 @@ eth_dev_null_create(const char *name,
eth_dev->dev_ops = &ops;
eth_dev->pci_dev = pci_dev;
eth_dev->driver = &rte_null_pmd;
+ TAILQ_INIT(&eth_dev->link_intr_cbs);

/* finally assign rx and tx ops */
if (packet_copy) {
@@ -461,6 +470,8 @@ error:
rte_free(data);
rte_free(pci_dev);
rte_free(internals);
+ rte_free(eth_dev);
+ rte_free(eth_drv);

return -1;
}
--
1.7.9.5
Tetsuya Mukawa
2015-09-29 02:24:26 UTC
Permalink
Post by Tomasz Kulasek
When device is added to the bonding, the link status callback is added to
the slave's eth_dev->link_intr_cbs list. This list is not initialized for
null pmd and adding it to the bonding segfaults application.
This patch allocates and sets up required structures.
---
drivers/net/null/rte_eth_null.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index e244595..a8b3191 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -386,6 +386,7 @@ eth_dev_null_create(const char *name,
const unsigned nb_rx_queues = 1;
const unsigned nb_tx_queues = 1;
struct rte_eth_dev_data *data = NULL;
+ struct eth_driver *eth_drv = NULL;
Hi Tomasz,

Thanks for extending null pmd features.
Is it possible to use rte_null_pmd here?
Could you please check ring pmd? It may also uses rte_ring_pmd for link
status callback.

Tetsuya
Kulasek, TomaszX
2015-09-29 09:39:39 UTC
Permalink
Hi Tetsuya,
Post by Tetsuya Mukawa
Thanks for extending null pmd features.
Is it possible to use rte_null_pmd here?
Could you please check ring pmd? It may also uses rte_ring_pmd for link
status callback.
Tetsuya
My first attempt was to use ring pmd, and there's no such an issue with it. It works pretty well in bonding.

Tomasz.
Tetsuya Mukawa
2015-09-29 10:32:58 UTC
Permalink
Post by Kulasek, TomaszX
Hi Tetsuya,
Post by Tetsuya Mukawa
Thanks for extending null pmd features.
Is it possible to use rte_null_pmd here?
Could you please check ring pmd? It may also uses rte_ring_pmd for link
status callback.
Tetsuya
My first attempt was to use ring pmd, and there's no such an issue with it. It works pretty well in bonding.
Tomasz.
HI Tomasz,


Sorry, my English is wrong.
'rte_null_pmd' is defined like below.

static struct eth_driver rte_null_pmd = {
.pci_drv = {
.name = "rte_null_pmd",
.drv_flags = RTE_PCI_DRV_DETACHABLE,
},
};

I guess you may be able to use 'rte_null_pmd' instead of allocating one
more eth_driver structure like below.

struct eth_driver *eth_drv = NULL;
eth_drv = rte_zmalloc_socket(name, sizeof(*eth_drv), 0, numa_node);

Is it possible to use 'rte_null_pmd'?

Tetsuya
Kulasek, TomaszX
2015-09-29 11:29:26 UTC
Permalink
-----Original Message-----
Sent: Tuesday, September 29, 2015 12:33
To: Kulasek, TomaszX
Subject: Re: [dpdk-dev] [PATCHv4 2/9] null: fix segfault when null_pmd
added to bonding
Post by Kulasek, TomaszX
Hi Tetsuya,
Post by Tetsuya Mukawa
Thanks for extending null pmd features.
Is it possible to use rte_null_pmd here?
Could you please check ring pmd? It may also uses rte_ring_pmd for
link status callback.
Tetsuya
My first attempt was to use ring pmd, and there's no such an issue
with it. It works pretty well in bonding.
Post by Kulasek, TomaszX
Tomasz.
HI Tomasz,
Sorry, my English is wrong.
'rte_null_pmd' is defined like below.
static struct eth_driver rte_null_pmd = {
.pci_drv = {
.name = "rte_null_pmd",
.drv_flags = RTE_PCI_DRV_DETACHABLE,
},
};
I guess you may be able to use 'rte_null_pmd' instead of allocating one
more eth_driver structure like below.
struct eth_driver *eth_drv = NULL;
eth_drv = rte_zmalloc_socket(name, sizeof(*eth_drv), 0, numa_node);
Is it possible to use 'rte_null_pmd'?
Tetsuya
Yes, you're right. This malloc can be removed.

Tomasz
Kulasek, TomaszX
2015-09-29 15:10:47 UTC
Permalink
Hi Tetsuya,

Changes from patch below, solves the problem. I've removed malloc.

Tomasz.

---
From e03e77a7dc0e47ac9d750545a834624f88f61966 Mon Sep 17 00:00:00 2001
From: Tomasz Kulasek <***@intel.com>
Date: Tue, 29 Sep 2015 16:48:31 +0200
Subject: [PATCH 1/4] null: fix segfault when null_pmd added to bonding

---
drivers/net/null/rte_eth_null.c | 3 +++
1 file changed, 3 insertions(+)

diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index e244595..b498ef1 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -432,6 +432,7 @@ eth_dev_null_create(const char *name,
internals->numa_node = numa_node;

pci_dev->numa_node = numa_node;
+ pci_dev->driver = &rte_null_pmd.pci_drv;

data->dev_private = internals;
data->port_id = eth_dev->data->port_id;
@@ -445,6 +446,7 @@ eth_dev_null_create(const char *name,
eth_dev->dev_ops = &ops;
eth_dev->pci_dev = pci_dev;
eth_dev->driver = &rte_null_pmd;
+ TAILQ_INIT(&eth_dev->link_intr_cbs);

/* finally assign rx and tx ops */
if (packet_copy) {
@@ -461,6 +463,7 @@ error:
rte_free(data);
rte_free(pci_dev);
rte_free(internals);
+ rte_free(eth_dev);

return -1;
}
--
Tetsuya Mukawa
2015-09-30 01:24:39 UTC
Permalink
Post by Kulasek, TomaszX
Hi Tetsuya,
Changes from patch below, solves the problem. I've removed malloc.
Tomasz.
---
From e03e77a7dc0e47ac9d750545a834624f88f61966 Mon Sep 17 00:00:00 2001
Date: Tue, 29 Sep 2015 16:48:31 +0200
Subject: [PATCH 1/4] null: fix segfault when null_pmd added to bonding
---
drivers/net/null/rte_eth_null.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index e244595..b498ef1 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -432,6 +432,7 @@ eth_dev_null_create(const char *name,
internals->numa_node = numa_node;
pci_dev->numa_node = numa_node;
+ pci_dev->driver = &rte_null_pmd.pci_drv;
data->dev_private = internals;
data->port_id = eth_dev->data->port_id;
@@ -445,6 +446,7 @@ eth_dev_null_create(const char *name,
eth_dev->dev_ops = &ops;
eth_dev->pci_dev = pci_dev;
eth_dev->driver = &rte_null_pmd;
+ TAILQ_INIT(&eth_dev->link_intr_cbs);
/* finally assign rx and tx ops */
if (packet_copy) {
rte_free(data);
rte_free(pci_dev);
rte_free(internals);
+ rte_free(eth_dev);
Hi Tomasz,

We can remove rte_free(eth_dev), because if eth_dev is allocated
correctly, we cannot reach here.
I don't see any issues except for it.

Thanks,
Tetsuya
Tomasz Kulasek
2015-07-15 17:26:23 UTC
Permalink
This patch adds a possibility to configure more than one queue on null
device.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/null/rte_eth_null.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)

diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index a8b3191..39ffcde 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -71,8 +71,8 @@ struct pmd_internals {
unsigned nb_rx_queues;
unsigned nb_tx_queues;

- struct null_queue rx_null_queues[1];
- struct null_queue tx_null_queues[1];
+ struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT];
+ struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT];
};


@@ -213,7 +213,7 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
if ((dev == NULL) || (mb_pool == NULL))
return -EINVAL;

- if (rx_queue_id != 0)
+ if (rx_queue_id >= dev->data->nb_rx_queues)
return -ENODEV;

internals = dev->data->dev_private;
@@ -246,7 +246,7 @@ eth_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id,
if (dev == NULL)
return -EINVAL;

- if (tx_queue_id != 0)
+ if (tx_queue_id >= dev->data->nb_tx_queues)
return -ENODEV;

internals = dev->data->dev_private;
@@ -279,8 +279,8 @@ eth_dev_info(struct rte_eth_dev *dev,
dev_info->driver_name = drivername;
dev_info->max_mac_addrs = 1;
dev_info->max_rx_pktlen = (uint32_t)-1;
- dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues;
- dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues;
+ dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues);
+ dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues);
dev_info->min_rx_bufsize = 0;
dev_info->pci_dev = NULL;
}
--
1.7.9.5
Tetsuya Mukawa
2015-09-29 02:24:39 UTC
Permalink
Post by Tomasz Kulasek
This patch adds a possibility to configure more than one queue on null
device.
---
drivers/net/null/rte_eth_null.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index a8b3191..39ffcde 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -71,8 +71,8 @@ struct pmd_internals {
unsigned nb_rx_queues;
unsigned nb_tx_queues;
- struct null_queue rx_null_queues[1];
- struct null_queue tx_null_queues[1];
+ struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT];
+ struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT];
};
@@ -213,7 +213,7 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
if ((dev == NULL) || (mb_pool == NULL))
return -EINVAL;
- if (rx_queue_id != 0)
+ if (rx_queue_id >= dev->data->nb_rx_queues)
return -ENODEV;
internals = dev->data->dev_private;
@@ -246,7 +246,7 @@ eth_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id,
if (dev == NULL)
return -EINVAL;
- if (tx_queue_id != 0)
+ if (tx_queue_id >= dev->data->nb_tx_queues)
return -ENODEV;
internals = dev->data->dev_private;
@@ -279,8 +279,8 @@ eth_dev_info(struct rte_eth_dev *dev,
dev_info->driver_name = drivername;
dev_info->max_mac_addrs = 1;
dev_info->max_rx_pktlen = (uint32_t)-1;
- dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues;
- dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues;
+ dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues);
+ dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues);
dev_info->min_rx_bufsize = 0;
dev_info->pci_dev = NULL;
}
Hi Thomasz,

To do like above, should we change below variables also?

static int
eth_dev_null_create(const char *name,
const unsigned numa_node,
unsigned packet_size,
unsigned packet_copy)
{
const unsigned nb_rx_queues = 1;
const unsigned nb_tx_queues = 1;

(snip)

}

Tetsuya
Kulasek, TomaszX
2015-09-29 09:46:11 UTC
Permalink
-----Original Message-----
Sent: Tuesday, September 29, 2015 04:25
Subject: Re: [dpdk-dev] [PATCHv4 3/9] null: extend number of virtual
queues
Post by Tomasz Kulasek
This patch adds a possibility to configure more than one queue on null
device.
---
drivers/net/null/rte_eth_null.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/net/null/rte_eth_null.c
b/drivers/net/null/rte_eth_null.c index a8b3191..39ffcde 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -71,8 +71,8 @@ struct pmd_internals {
unsigned nb_rx_queues;
unsigned nb_tx_queues;
- struct null_queue rx_null_queues[1];
- struct null_queue tx_null_queues[1];
+ struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT];
+ struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT];
};
@@ -213,7 +213,7 @@ eth_rx_queue_setup(struct rte_eth_dev *dev,
uint16_t rx_queue_id,
Post by Tomasz Kulasek
if ((dev == NULL) || (mb_pool == NULL))
return -EINVAL;
- if (rx_queue_id != 0)
+ if (rx_queue_id >= dev->data->nb_rx_queues)
return -ENODEV;
internals = dev->data->dev_private;
@@ -246,7 +246,7 @@ eth_tx_queue_setup(struct rte_eth_dev *dev,
uint16_t tx_queue_id,
Post by Tomasz Kulasek
if (dev == NULL)
return -EINVAL;
- if (tx_queue_id != 0)
+ if (tx_queue_id >= dev->data->nb_tx_queues)
return -ENODEV;
internals = dev->data->dev_private;
@@ -279,8 +279,8 @@ eth_dev_info(struct rte_eth_dev *dev,
dev_info->driver_name = drivername;
dev_info->max_mac_addrs = 1;
dev_info->max_rx_pktlen = (uint32_t)-1;
- dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues;
- dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues;
+ dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues);
+ dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues);
dev_info->min_rx_bufsize = 0;
dev_info->pci_dev = NULL;
}
Hi Thomasz,
To do like above, should we change below variables also?
static int
eth_dev_null_create(const char *name,
const unsigned numa_node,
unsigned packet_size,
unsigned packet_copy)
{
const unsigned nb_rx_queues = 1;
const unsigned nb_tx_queues = 1;
(snip)
}
Tetsuya
Hi Tetsuya,

The number of requested queues are configured later, through the rte eth_dev_configure API. Here these values mean the default numbe
Tetsuya Mukawa
2015-09-29 10:34:28 UTC
Permalink
Post by Kulasek, TomaszX
-----Original Message-----
Sent: Tuesday, September 29, 2015 04:25
Subject: Re: [dpdk-dev] [PATCHv4 3/9] null: extend number of virtual
queues
Post by Tomasz Kulasek
This patch adds a possibility to configure more than one queue on null
device.
---
drivers/net/null/rte_eth_null.c | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/drivers/net/null/rte_eth_null.c
b/drivers/net/null/rte_eth_null.c index a8b3191..39ffcde 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -71,8 +71,8 @@ struct pmd_internals {
unsigned nb_rx_queues;
unsigned nb_tx_queues;
- struct null_queue rx_null_queues[1];
- struct null_queue tx_null_queues[1];
+ struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT];
+ struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT];
};
@@ -213,7 +213,7 @@ eth_rx_queue_setup(struct rte_eth_dev *dev,
uint16_t rx_queue_id,
Post by Tomasz Kulasek
if ((dev == NULL) || (mb_pool == NULL))
return -EINVAL;
- if (rx_queue_id != 0)
+ if (rx_queue_id >= dev->data->nb_rx_queues)
return -ENODEV;
internals = dev->data->dev_private;
@@ -246,7 +246,7 @@ eth_tx_queue_setup(struct rte_eth_dev *dev,
uint16_t tx_queue_id,
Post by Tomasz Kulasek
if (dev == NULL)
return -EINVAL;
- if (tx_queue_id != 0)
+ if (tx_queue_id >= dev->data->nb_tx_queues)
return -ENODEV;
internals = dev->data->dev_private;
@@ -279,8 +279,8 @@ eth_dev_info(struct rte_eth_dev *dev,
dev_info->driver_name = drivername;
dev_info->max_mac_addrs = 1;
dev_info->max_rx_pktlen = (uint32_t)-1;
- dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues;
- dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues;
+ dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues);
+ dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues);
dev_info->min_rx_bufsize = 0;
dev_info->pci_dev = NULL;
}
Hi Thomasz,
To do like above, should we change below variables also?
static int
eth_dev_null_create(const char *name,
const unsigned numa_node,
unsigned packet_size,
unsigned packet_copy)
{
const unsigned nb_rx_queues = 1;
const unsigned nb_tx_queues = 1;
(snip)
}
Tetsuya
Hi Tetsuya,
The number of requested queues are configured later, through the rte eth_dev_configure API. Here these values mean the default number of queues after null pmd creation.
Tomasz.
Hi Tomasz,

I guess we may need to change below variables when a queue is configured.
- internals->nb_rx_queues;
- internals->nb_tx_queues;
Without changing, 'num_stats' in eth_stats_get() might be wrong value.

Tetsuya
Kulasek, TomaszX
2015-09-29 11:56:49 UTC
Permalink
Post by Thomas Monjalon
Hi Tomasz,
I guess we may need to change below variables when a queue is
configured.
- internals->nb_rx_queues;
- internals->nb_tx_queues;
Without changing, 'num_stats' in eth_stats_get() might be wrong value.
Tetsuya
Hi Tetsuya,

I see it now. I was too focused on one case and I missed it. I'll modify
Kulasek, TomaszX
2015-09-29 15:06:15 UTC
Permalink
Hi Tetsuya,

Can you check patch below?

Thanks,
Tomasz.
---

From d50b0594fa34a576602c457b51cac80e7d462eed Mon Sep 17 00:00:00 2001
From: Tomasz Kulasek <***@intel.com>
Date: Tue, 29 Sep 2015 16:52:12 +0200
Subject: [PATCH 2/4] null: extend number of virtual queues

---
drivers/net/null/rte_eth_null.c | 28 +++++++++++++++++++---------
1 file changed, 19 insertions(+), 9 deletions(-)

diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index b498ef1..68cb723 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -71,8 +71,8 @@ struct pmd_internals {
unsigned nb_rx_queues;
unsigned nb_tx_queues;

- struct null_queue rx_null_queues[1];
- struct null_queue tx_null_queues[1];
+ struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT];
+ struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT];
};


@@ -178,7 +178,15 @@ eth_null_copy_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
}

static int
-eth_dev_configure(struct rte_eth_dev *dev __rte_unused) { return 0; }
+eth_dev_configure(struct rte_eth_dev *dev) {
+ struct pmd_internals *internals;
+
+ internals = dev->data->dev_private;
+ internals->nb_rx_queues = dev->data->nb_rx_queues;
+ internals->nb_tx_queues = dev->data->nb_tx_queues;
+
+ return 0;
+}

static int
eth_dev_start(struct rte_eth_dev *dev)
@@ -213,10 +221,11 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
if ((dev == NULL) || (mb_pool == NULL))
return -EINVAL;

- if (rx_queue_id != 0)
+ internals = dev->data->dev_private;
+
+ if (rx_queue_id >= internals->nb_rx_queues)
return -ENODEV;

- internals = dev->data->dev_private;
packet_size = internals->packet_size;

internals->rx_null_queues[rx_queue_id].mb_pool = mb_pool;
@@ -246,10 +255,11 @@ eth_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id,
if (dev == NULL)
return -EINVAL;

- if (tx_queue_id != 0)
+ internals = dev->data->dev_private;
+
+ if (tx_queue_id >= internals->nb_tx_queues)
return -ENODEV;

- internals = dev->data->dev_private;
packet_size = internals->packet_size;

dev->data->tx_queues[tx_queue_id] =
@@ -279,8 +289,8 @@ eth_dev_info(struct rte_eth_dev *dev,
dev_info->driver_name = drivername;
dev_info->max_mac_addrs = 1;
dev_info->max_rx_pktlen = (uint32_t)-1;
- dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues;
- dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues;
+ dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues);
+ dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues);
dev_info->min_rx_bufsize = 0;
dev_info->pci_dev = NULL;
}
Tetsuya Mukawa
2015-09-30 01:24:44 UTC
Permalink
Post by Kulasek, TomaszX
Hi Tetsuya,
Can you check patch below?
Thanks,
Tomasz.
---
From d50b0594fa34a576602c457b51cac80e7d462eed Mon Sep 17 00:00:00 2001
Date: Tue, 29 Sep 2015 16:52:12 +0200
Subject: [PATCH 2/4] null: extend number of virtual queues
---
drivers/net/null/rte_eth_null.c | 28 +++++++++++++++++++---------
1 file changed, 19 insertions(+), 9 deletions(-)
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index b498ef1..68cb723 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -71,8 +71,8 @@ struct pmd_internals {
unsigned nb_rx_queues;
unsigned nb_tx_queues;
- struct null_queue rx_null_queues[1];
- struct null_queue tx_null_queues[1];
+ struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT];
+ struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT];
};
@@ -178,7 +178,15 @@ eth_null_copy_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
}
static int
-eth_dev_configure(struct rte_eth_dev *dev __rte_unused) { return 0; }
+eth_dev_configure(struct rte_eth_dev *dev) {
+ struct pmd_internals *internals;
+
+ internals = dev->data->dev_private;
+ internals->nb_rx_queues = dev->data->nb_rx_queues;
+ internals->nb_tx_queues = dev->data->nb_tx_queues;
+
+ return 0;
+}
static int
eth_dev_start(struct rte_eth_dev *dev)
@@ -213,10 +221,11 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
if ((dev == NULL) || (mb_pool == NULL))
return -EINVAL;
- if (rx_queue_id != 0)
+ internals = dev->data->dev_private;
+
+ if (rx_queue_id >= internals->nb_rx_queues)
return -ENODEV;
- internals = dev->data->dev_private;
packet_size = internals->packet_size;
internals->rx_null_queues[rx_queue_id].mb_pool = mb_pool;
@@ -246,10 +255,11 @@ eth_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id,
if (dev == NULL)
return -EINVAL;
- if (tx_queue_id != 0)
+ internals = dev->data->dev_private;
+
+ if (tx_queue_id >= internals->nb_tx_queues)
return -ENODEV;
- internals = dev->data->dev_private;
packet_size = internals->packet_size;
dev->data->tx_queues[tx_queue_id] =
@@ -279,8 +289,8 @@ eth_dev_info(struct rte_eth_dev *dev,
dev_info->driver_name = drivername;
dev_info->max_mac_addrs = 1;
dev_info->max_rx_pktlen = (uint32_t)-1;
- dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues;
- dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues;
+ dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues);
+ dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues);
dev_info->min_rx_bufsize = 0;
dev_info->pci_dev = NULL;
}
--
Acked-by: Tetsuya Mukawa <***@igel.co.jp>
Tomasz Kulasek
2015-07-15 17:26:24 UTC
Permalink
This implementation allows to set and read RSS configuration for null
device, and is used to validate right values propagation over the slaves,
in test units for dynamic RSS configuration for bonding.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/null/rte_eth_null.c | 116 +++++++++++++++++++++++++++++++++++++++
1 file changed, 116 insertions(+)

diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 39ffcde..f393422 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -37,6 +37,8 @@
#include <rte_memcpy.h>
#include <rte_dev.h>
#include <rte_kvargs.h>
+#include <rte_eth_null.h>
+#include <rte_spinlock.h>

#define ETH_NULL_PACKET_SIZE_ARG "size"
#define ETH_NULL_PACKET_COPY_ARG "copy"
@@ -73,6 +75,17 @@ struct pmd_internals {

struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT];
struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT];
+
+ /** Bit mask of RSS offloads, the bit offset also means flow type */
+ uint64_t flow_type_rss_offloads;
+
+ rte_spinlock_t rss_lock;
+
+ uint16_t reta_size;
+ struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_128 /
+ RTE_RETA_GROUP_SIZE];
+
+ uint8_t rss_key[40]; /**< 40-byte hash key. */
};


@@ -283,6 +296,8 @@ eth_dev_info(struct rte_eth_dev *dev,
dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues);
dev_info->min_rx_bufsize = 0;
dev_info->pci_dev = NULL;
+ dev_info->reta_size = internals->reta_size;
+ dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads;
}

static void
@@ -363,6 +378,91 @@ static int
eth_link_update(struct rte_eth_dev *dev __rte_unused,
int wait_to_complete __rte_unused) { return 0; }

+static int
+eth_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ int i, j;
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ if (reta_size != internal->reta_size)
+ return -EINVAL;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ /* Copy RETA table */
+ for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) {
+ internal->reta_conf[i].mask = reta_conf[i].mask;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ internal->reta_conf[i].reta[j] = reta_conf[i].reta[j];
+ }
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ int i, j;
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ if (reta_size != internal->reta_size)
+ return -EINVAL;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ /* Copy RETA table */
+ for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) {
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ reta_conf[i].reta[j] = internal->reta_conf[i].reta[j];
+ }
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_hash_update(struct rte_eth_dev *dev, struct rte_eth_rss_conf *rss_conf)
+{
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0)
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ rss_conf->rss_hf & internal->flow_type_rss_offloads;
+
+ if (rss_conf->rss_key)
+ memcpy(internal->rss_key, rss_conf->rss_key, 40);
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ if (rss_conf->rss_key)
+ memcpy(rss_conf->rss_key, internal->rss_key, 40);
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
static const struct eth_dev_ops ops = {
.dev_start = eth_dev_start,
.dev_stop = eth_dev_stop,
@@ -375,6 +475,10 @@ static const struct eth_dev_ops ops = {
.link_update = eth_link_update,
.stats_get = eth_stats_get,
.stats_reset = eth_stats_reset,
+ .reta_update = eth_rss_reta_update,
+ .reta_query = eth_rss_reta_query,
+ .rss_hash_update = eth_rss_hash_update,
+ .rss_hash_conf_get = eth_rss_hash_conf_get
};

static int
@@ -391,6 +495,13 @@ eth_dev_null_create(const char *name,
struct pmd_internals *internals = NULL;
struct rte_eth_dev *eth_dev = NULL;

+ static const uint8_t default_rss_key[40] = {
+ 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D,
+ 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+ 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B,
+ 0xBE, 0xAC, 0x01, 0xFA
+ };
+
if (name == NULL)
return -EINVAL;

@@ -436,6 +547,11 @@ eth_dev_null_create(const char *name,
internals->packet_copy = packet_copy;
internals->numa_node = numa_node;

+ internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK;
+ internals->reta_size = RTE_DIM(internals->reta_conf) * RTE_RETA_GROUP_SIZE;
+
+ memcpy(internals->rss_key, default_rss_key, 40);
+
eth_drv->pci_drv.name = drivername;

pci_dev->numa_node = numa_node;
--
1.7.9.5
Tetsuya Mukawa
2015-09-29 02:24:51 UTC
Permalink
Post by Tomasz Kulasek
This implementation allows to set and read RSS configuration for null
device, and is used to validate right values propagation over the slaves,
in test units for dynamic RSS configuration for bonding.
---
drivers/net/null/rte_eth_null.c | 116 +++++++++++++++++++++++++++++++++++++++
1 file changed, 116 insertions(+)
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 39ffcde..f393422 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
+static int
+eth_rss_hash_update(struct rte_eth_dev *dev, struct rte_eth_rss_conf *rss_conf)
+{
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0)
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ rss_conf->rss_hf & internal->flow_type_rss_offloads;
+
+ if (rss_conf->rss_key)
+ memcpy(internal->rss_key, rss_conf->rss_key, 40);
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ if (rss_conf->rss_key)
+ memcpy(rss_conf->rss_key, internal->rss_key, 40);
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
static const struct eth_dev_ops ops = {
.dev_start = eth_dev_start,
.dev_stop = eth_dev_stop,
@@ -436,6 +547,11 @@ eth_dev_null_create(const char *name,
internals->packet_copy = packet_copy;
internals->numa_node = numa_node;
+ internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK;
+ internals->reta_size = RTE_DIM(internals->reta_conf) * RTE_RETA_GROUP_SIZE;
+
+ memcpy(internals->rss_key, default_rss_key, 40);
+
eth_drv->pci_drv.name = drivername;
pci_dev->numa_node = numa_node;
Hi Thomasz,

I am just curious. Is it possible to use rte_memcpy instead of memcpy?
if we can, rte_memcpy may be faster.

Tetsuya
Kulasek, TomaszX
2015-09-29 10:04:14 UTC
Permalink
Post by Tetsuya Mukawa
Hi Thomasz,
I am just curious. Is it possible to use rte_memcpy instead of memcpy?
if we can, rte_memcpy may be faster.
Tetsuya
Hi Tetsuya,

I've just tested it and seems to work, so there's no objections to use rte_memcpy.

Tomasz.
Jastrzebski, MichalX K
2015-10-12 09:05:33 UTC
Permalink
-----Original Message-----
Sent: Tuesday, September 29, 2015 4:25 AM
Subject: Re: [dpdk-dev] [PATCHv4 4/9] null: virtual dynamic rss configuration
Post by Tomasz Kulasek
This implementation allows to set and read RSS configuration for null
device, and is used to validate right values propagation over the slaves,
in test units for dynamic RSS configuration for bonding.
---
drivers/net/null/rte_eth_null.c | 116
+++++++++++++++++++++++++++++++++++++++
Post by Tomasz Kulasek
1 file changed, 116 insertions(+)
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index 39ffcde..f393422 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
+static int
+eth_rss_hash_update(struct rte_eth_dev *dev, struct rte_eth_rss_conf
*rss_conf)
Post by Tomasz Kulasek
+{
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0)
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ rss_conf->rss_hf & internal-
flow_type_rss_offloads;
+
+ if (rss_conf->rss_key)
+ memcpy(internal->rss_key, rss_conf->rss_key, 40);
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ rss_conf->rss_hf = dev->data-
dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ if (rss_conf->rss_key)
+ memcpy(rss_conf->rss_key, internal->rss_key, 40);
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
static const struct eth_dev_ops ops = {
.dev_start = eth_dev_start,
.dev_stop = eth_dev_stop,
@@ -436,6 +547,11 @@ eth_dev_null_create(const char *name,
internals->packet_copy = packet_copy;
internals->numa_node = numa_node;
+ internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK;
+ internals->reta_size = RTE_DIM(internals->reta_conf) *
RTE_RETA_GROUP_SIZE;
Post by Tomasz Kulasek
+
+ memcpy(internals->rss_key, default_rss_key, 40);
+
eth_drv->pci_drv.name = drivername;
pci_dev->numa_node = numa_node;
Hi Thomasz,
I am just curious. Is it possible to use rte_memcpy instead of memcpy?
if we can, rte_memcpy may be faster.
Tetsuya
Hi Tetsuya,
Could You please review v5 that Tomasz sent and if You agree with this implementation could You also ACK this patch-set?
Tomasz Kulasek
2015-07-15 17:26:25 UTC
Permalink
To use eth_dev_null_create in application this method needs to be exported.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/null/Makefile | 2 +-
drivers/net/null/rte_eth_null.c | 3 ++-
drivers/net/null/rte_eth_null.h | 40 +++++++++++++++++++++++++++++
drivers/net/null/rte_pmd_null_version.map | 7 +++++
4 files changed, 50 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/null/rte_eth_null.h

diff --git a/drivers/net/null/Makefile b/drivers/net/null/Makefile
index 6472015..b33f9fd 100644
--- a/drivers/net/null/Makefile
+++ b/drivers/net/null/Makefile
@@ -51,7 +51,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += rte_eth_null.c
#
# Export include files
#
-SYMLINK-y-include +=
+SYMLINK-y-include += rte_eth_null.h

# this lib depends upon:
DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_mbuf
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index f393422..8ae6ebf 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -2,6 +2,7 @@
* BSD LICENSE
*
* Copyright (C) IGEL Co.,Ltd.
+ * Copyright (c) 2015 Intel Corporation.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -481,7 +482,7 @@ static const struct eth_dev_ops ops = {
.rss_hash_conf_get = eth_rss_hash_conf_get
};

-static int
+int
eth_dev_null_create(const char *name,
const unsigned numa_node,
unsigned packet_size,
diff --git a/drivers/net/null/rte_eth_null.h b/drivers/net/null/rte_eth_null.h
new file mode 100644
index 0000000..abada8c
--- /dev/null
+++ b/drivers/net/null/rte_eth_null.h
@@ -0,0 +1,40 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef RTE_ETH_NULL_H_
+#define RTE_ETH_NULL_H_
+
+int eth_dev_null_create(const char *name, const unsigned numa_node,
+ unsigned packet_size, unsigned packet_copy);
+
+#endif /* RTE_ETH_NULL_H_ */
diff --git a/drivers/net/null/rte_pmd_null_version.map b/drivers/net/null/rte_pmd_null_version.map
index ef35398..2b2a743 100644
--- a/drivers/net/null/rte_pmd_null_version.map
+++ b/drivers/net/null/rte_pmd_null_version.map
@@ -2,3 +2,10 @@ DPDK_2.0 {

local: *;
};
+
+DPDK_2.1 {
+ global:
+
+ eth_dev_null_create;
+
+} DPDK_2.0;
--
1.7.9.5
Thomas Monjalon
2015-09-25 10:11:10 UTC
Permalink
Hi Tomasz,
Post by Tomasz Kulasek
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -2,6 +2,7 @@
* BSD LICENSE
*
* Copyright (C) IGEL Co.,Ltd.
+ * Copyright (c) 2015 Intel Corporation.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -481,7 +482,7 @@ static const struct eth_dev_ops ops = {
.rss_hash_conf_get = eth_rss_hash_conf_get
};
-static int
+int
eth_dev_null_create(const char *name,
const unsigned numa_node,
unsigned packet_size,
There is no rule to update a copyright but I think it is not justified here.
Post by Tomasz Kulasek
--- a/drivers/net/null/rte_pmd_null_version.map
+++ b/drivers/net/null/rte_pmd_null_version.map
@@ -2,3 +2,10 @@ DPDK_2.0 {
local: *;
};
+
+DPDK_2.1 {
+
+ eth_dev_null_create;
+
+} DPDK_2.0;
Please, could you update it to DPDK_2.2?

Thanks
Kulasek, TomaszX
2015-09-25 13:24:28 UTC
Permalink
Hi Thomas,
-----Original Message-----
Sent: Friday, September 25, 2015 12:11
To: Kulasek, TomaszX
Subject: Re: [dpdk-dev] [PATCHv4 5/9] null: export eth_dev_null_create
Post by Tomasz Kulasek
--- a/drivers/net/null/rte_pmd_null_version.map
+++ b/drivers/net/null/rte_pmd_null_version.map
@@ -2,3 +2,10 @@ DPDK_2.0 {
local: *;
};
+
+DPDK_2.1 {
+
+ eth_dev_null_create;
+
+} DPDK_2.0;
Please, could you update it to DPDK_2.2?
Thanks
--- a/drivers/net/null/rte_pmd_null_version.map
+++ b/drivers/net/null/rte_pmd_null_version.map
@@ -2,3 +2,10 @@ DPDK_2.0 {

local: *;
};
+
+DPDK_2.2 {
+ global:
+
+ eth_dev_null_create;
+
+} DPDK_2.0;

This change is fine?

Tomasz.
Thomas Monjalon
2015-09-25 13:43:22 UTC
Permalink
Post by Kulasek, TomaszX
Post by Thomas Monjalon
Please, could you update it to DPDK_2.2?
+DPDK_2.2 {
+
+ eth_dev_null_create;
+
+} DPDK_2.0;
This change is fine?
Yes, only one letter to change :)
Tetsuya Mukawa
2015-09-29 02:28:46 UTC
Permalink
Post by Tomasz Kulasek
To use eth_dev_null_create in application this method needs to be exported.
---
drivers/net/null/Makefile | 2 +-
drivers/net/null/rte_eth_null.c | 3 ++-
drivers/net/null/rte_eth_null.h | 40 +++++++++++++++++++++++++++++
drivers/net/null/rte_pmd_null_version.map | 7 +++++
4 files changed, 50 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/null/rte_eth_null.h
diff --git a/drivers/net/null/Makefile b/drivers/net/null/Makefile
index 6472015..b33f9fd 100644
--- a/drivers/net/null/Makefile
+++ b/drivers/net/null/Makefile
@@ -51,7 +51,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += rte_eth_null.c
#
# Export include files
#
-SYMLINK-y-include +=
+SYMLINK-y-include += rte_eth_null.h
DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_mbuf
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index f393422..8ae6ebf 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -2,6 +2,7 @@
* BSD LICENSE
*
* Copyright (C) IGEL Co.,Ltd.
+ * Copyright (c) 2015 Intel Corporation.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
@@ -481,7 +482,7 @@ static const struct eth_dev_ops ops = {
.rss_hash_conf_get = eth_rss_hash_conf_get
};
-static int
+int
eth_dev_null_create(const char *name,
const unsigned numa_node,
unsigned packet_size,
diff --git a/drivers/net/null/rte_eth_null.h b/drivers/net/null/rte_eth_null.h
new file mode 100644
index 0000000..abada8c
--- /dev/null
+++ b/drivers/net/null/rte_eth_null.h
@@ -0,0 +1,40 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef RTE_ETH_NULL_H_
+#define RTE_ETH_NULL_H_
+
+int eth_dev_null_create(const char *name, const unsigned numa_node,
+ unsigned packet_size, unsigned packet_copy);
+
+#endif /* RTE_ETH_NULL_H_ */
diff --git a/drivers/net/null/rte_pmd_null_version.map b/drivers/net/null/rte_pmd_null_version.map
index ef35398..2b2a743 100644
--- a/drivers/net/null/rte_pmd_null_version.map
+++ b/drivers/net/null/rte_pmd_null_version.map
@@ -2,3 +2,10 @@ DPDK_2.0 {
local: *;
};
+
+DPDK_2.1 {
+
+ eth_dev_null_create;
+
+} DPDK_2.0;
Hi Thomasz,

I don't have any comments without the version miss Thomas has already
commented.

Tetsuya
Tomasz Kulasek
2015-07-15 17:26:26 UTC
Permalink
This test module uses modified null device to check right RSS configuration
propagation.

1) Propagation test
a) Set RSS hash function for bonding, fetch RSS hash function from
bonded slave, check if same. Do it for all slaves.
b) Repeat above for RSS key and RETA.

2)Dynamic adding slave to the bonding
a) Remove slave from bonding.
b) Change its configuration to other than bonding.
c) Add slave to the started bonding device.
d) Check if slaves configuration changed.

3) Repeat point 1) and 2) with RSS multi-queue mode enabled/disabled for
bonding port.

v4 changes:
- adapted to use null pmd driver in place of ring pmd

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
app/test/Makefile | 8 +
app/test/test_link_bonding_rssconf.c | 679 ++++++++++++++++++++++++++++++++++
2 files changed, 687 insertions(+)
create mode 100644 app/test/test_link_bonding_rssconf.c

diff --git a/app/test/Makefile b/app/test/Makefile
index caa359c..4d87d85 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -137,6 +137,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_ACL) += test_acl.c
ifeq ($(CONFIG_RTE_LIBRTE_PMD_RING),y)
SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_mode4.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_rssconf.c
endif

SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring.c
@@ -173,6 +174,13 @@ ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
LDLIBS += -lrte_pmd_ring
endif
endif
+ifneq ($(CONFIG_RTE_LIBRTE_PMD_NULL),y)
+$(error Link bonding rssconf tests require CONFIG_RTE_LIBRTE_PMD_NULL=y)
+else
+ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
+LDLIBS += -lrte_pmd_null
+endif
+endif
endif

include $(RTE_SDK)/mk/rte.app.mk
diff --git a/app/test/test_link_bonding_rssconf.c b/app/test/test_link_bonding_rssconf.c
new file mode 100644
index 0000000..e6714b4
--- /dev/null
+++ b/app/test/test_link_bonding_rssconf.c
@@ -0,0 +1,679 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <errno.h>
+#include <rte_cycles.h>
+#include <sys/queue.h>
+
+#include <rte_byteorder.h>
+#include <rte_common.h>
+#include <rte_debug.h>
+#include <rte_ethdev.h>
+#include <rte_log.h>
+#include <rte_lcore.h>
+#include <rte_memory.h>
+
+#include <rte_string_fns.h>
+#include <rte_errno.h>
+#include <rte_eth_bond.h>
+#include <rte_eth_null.h>
+
+#include "test.h"
+
+#define SLAVE_COUNT (4)
+
+#define RXTX_RING_SIZE 1024
+#define RXTX_QUEUE_COUNT 4
+
+#define BONDED_DEV_NAME ("rssconf_bond_dev")
+
+#define SLAVE_DEV_NAME_FMT ("rssconf_slave%d")
+#define SLAVE_RXTX_QUEUE_FMT ("rssconf_slave%d_q%d")
+
+#define NUM_MBUFS 8191
+#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)
+#define MBUF_CACHE_SIZE 250
+#define BURST_SIZE 32
+
+#define INVALID_SOCKET_ID (-1)
+#define INVALID_PORT_ID (0xFF)
+#define INVALID_BONDING_MODE (-1)
+
+struct slave_conf {
+ uint8_t port_id;
+ struct rte_eth_dev_info dev_info;
+
+ struct rte_eth_rss_conf rss_conf;
+ uint8_t rss_key[40];
+ struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE];
+
+ uint8_t is_slave;
+ struct rte_ring *rxtx_queue[RXTX_QUEUE_COUNT];
+};
+
+struct link_bonding_rssconf_unittest_params {
+ uint8_t bond_port_id;
+ struct rte_eth_dev_info bond_dev_info;
+ struct rte_eth_rss_reta_entry64 bond_reta_conf[512 / RTE_RETA_GROUP_SIZE];
+ struct slave_conf slave_ports[SLAVE_COUNT];
+
+ struct rte_mempool *mbuf_pool;
+};
+
+static struct link_bonding_rssconf_unittest_params test_params = {
+ .bond_port_id = INVALID_PORT_ID,
+ .slave_ports = {
+ [0 ... SLAVE_COUNT - 1] = { .port_id = INVALID_PORT_ID, .is_slave = 0}
+ },
+ .mbuf_pool = NULL,
+};
+
+/**
+ * Default port configuration with RSS turned off
+ */
+static struct rte_eth_conf default_pmd_conf = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_NONE,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled */
+ .hw_ip_checksum = 0, /**< IP checksum offload enabled */
+ .hw_vlan_filter = 0, /**< VLAN filtering disabled */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled */
+ .hw_strip_crc = 0, /**< CRC stripped by hardware */
+ },
+ .txmode = {
+ .mq_mode = ETH_MQ_TX_NONE,
+ },
+ .lpbk_mode = 0,
+};
+
+static struct rte_eth_conf rss_pmd_conf = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_RSS,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled */
+ .hw_ip_checksum = 0, /**< IP checksum offload enabled */
+ .hw_vlan_filter = 0, /**< VLAN filtering disabled */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled */
+ .hw_strip_crc = 0, /**< CRC stripped by hardware */
+ },
+ .txmode = {
+ .mq_mode = ETH_MQ_TX_NONE,
+ },
+ .rx_adv_conf = {
+ .rss_conf = {
+ .rss_key = NULL,
+ .rss_hf = ETH_RSS_IPV6,
+ },
+ },
+ .lpbk_mode = 0,
+};
+
+#define FOR_EACH(_i, _item, _array, _size) \
+ for (_i = 0, _item = &_array[0]; _i < _size && (_item = &_array[_i]); _i++)
+
+/* Macro for iterating over every port that can be used as a slave
+ * in this test.
+ * _i variable used as an index in test_params->slave_ports
+ * _slave pointer to &test_params->slave_ports[_idx]
+ */
+#define FOR_EACH_PORT(_i, _port) \
+ FOR_EACH(_i, _port, test_params.slave_ports, \
+ RTE_DIM(test_params.slave_ports))
+
+static int
+configure_ethdev(uint8_t port_id, struct rte_eth_conf *eth_conf, uint8_t start)
+{
+ int rxq, txq;
+
+ TEST_ASSERT(rte_eth_dev_configure(port_id, RXTX_QUEUE_COUNT,
+ RXTX_QUEUE_COUNT, eth_conf) == 0, "Failed to configure device %u",
+ port_id);
+
+ for (rxq = 0; rxq < RXTX_QUEUE_COUNT; rxq++) {
+ TEST_ASSERT(rte_eth_rx_queue_setup(port_id, rxq, RXTX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id), NULL,
+ test_params.mbuf_pool) == 0, "Failed to setup rx queue.");
+ }
+
+ for (txq = 0; txq < RXTX_QUEUE_COUNT; txq++) {
+ TEST_ASSERT(rte_eth_tx_queue_setup(port_id, txq, RXTX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id), NULL) == 0,
+ "Failed to setup tx queue.");
+ }
+
+ if (start) {
+ TEST_ASSERT(rte_eth_dev_start(port_id) == 0,
+ "Failed to start device (%d).", port_id);
+ }
+
+ return 0;
+}
+
+/**
+ * Remove all slaves from bonding
+ */
+static int
+remove_slaves(void)
+{
+ unsigned n;
+ struct slave_conf *port;
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ if (port->is_slave) {
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(
+ test_params.bond_port_id, port->port_id),
+ "Cannot remove slave %d from bonding", port->port_id);
+ port->is_slave = 0;
+ }
+ }
+
+ return 0;
+}
+
+static int
+remove_slaves_and_stop_bonded_device(void)
+{
+ TEST_ASSERT_SUCCESS(remove_slaves(), "Removing slaves");
+ rte_eth_dev_stop(test_params.bond_port_id);
+ return TEST_SUCCESS;
+}
+
+/**
+ * Add all slaves to bonding
+ */
+static int
+bond_slaves(void)
+{
+ unsigned n;
+ struct slave_conf *port;
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ if (!port->is_slave) {
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id,
+ port->port_id), "Cannot attach slave %d to the bonding",
+ port->port_id);
+ port->is_slave = 1;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * Set all RETA values in port_id to value
+ */
+static int
+reta_set(uint8_t port_id, uint8_t value, int reta_size)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ int i, j;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) {
+ /* select all fields to set */
+ reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ reta_conf[i].reta[j] = value;
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
+
+/**
+ * Check if slaves RETA is synchronized with bonding port. Returns 1 if slave
+ * port is synced with bonding port.
+ */
+static int
+reta_check_synced(struct slave_conf *port)
+{
+ unsigned i;
+
+ for (i = 0; i < test_params.bond_dev_info.reta_size;
+ i++) {
+
+ int index = i / RTE_RETA_GROUP_SIZE;
+ int shift = i % RTE_RETA_GROUP_SIZE;
+
+ if (port->reta_conf[index].reta[shift] !=
+ test_params.bond_reta_conf[index].reta[shift])
+ return 0;
+
+ }
+
+ return 1;
+}
+
+/**
+ * Fetch bonding ports RETA
+ */
+static int
+bond_reta_fetch(void) {
+ unsigned j;
+
+ for (j = 0; j < test_params.bond_dev_info.reta_size / RTE_RETA_GROUP_SIZE;
+ j++)
+ test_params.bond_reta_conf[j].mask = ~0LL;
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(test_params.bond_port_id,
+ test_params.bond_reta_conf, test_params.bond_dev_info.reta_size),
+ "Cannot take bonding ports RSS configuration");
+ return 0;
+}
+
+/**
+ * Fetch slaves RETA
+ */
+static int
+slave_reta_fetch(struct slave_conf *port) {
+ unsigned j;
+
+ for (j = 0; j < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; j++)
+ port->reta_conf[j].mask = ~0LL;
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(port->port_id,
+ port->reta_conf, port->dev_info.reta_size),
+ "Cannot take bonding ports RSS configuration");
+ return 0;
+}
+
+/**
+ * Remove and add slave to check if slaves configuration is synced with
+ * the bonding ports values after adding new slave.
+ */
+static int
+slave_remove_and_add(void)
+{
+ struct slave_conf *port = &(test_params.slave_ports[0]);
+
+ /* 1. Remove first slave from bonding */
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(test_params.bond_port_id,
+ port->port_id), "Cannot remove slave #d from bonding");
+
+ /* 2. Change removed (ex-)slave and bonding configuration to different
+ * values
+ */
+ reta_set(test_params.bond_port_id, 1, test_params.bond_dev_info.reta_size);
+ bond_reta_fetch();
+
+ reta_set(port->port_id, 2, port->dev_info.reta_size);
+ slave_reta_fetch(port);
+
+ TEST_ASSERT(reta_check_synced(port) == 0,
+ "Removed slave didn't should be synchronized with bonding port");
+
+ /* 3. Add (ex-)slave and check if configuration changed*/
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id,
+ port->port_id), "Cannot add slave");
+
+ bond_reta_fetch();
+ slave_reta_fetch(port);
+
+ return reta_check_synced(port);
+}
+
+/**
+ * Test configuration propagation over slaves.
+ */
+static int
+test_propagate(void)
+{
+ unsigned i;
+ uint8_t n;
+ struct slave_conf *port;
+ uint8_t bond_rss_key[40];
+ struct rte_eth_rss_conf bond_rss_conf;
+
+ int retval = 0;
+ uint64_t rss_hf = 0;
+ uint64_t default_rss_hf = 0;
+
+ rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info);
+
+ /*
+ * Test hash function propagation
+ */
+ for (i = 0; i < sizeof(test_params.bond_dev_info.flow_type_rss_offloads)*8;
+ i++) {
+
+ rss_hf = test_params.bond_dev_info.flow_type_rss_offloads & (1<<i);
+ if (rss_hf) {
+ bond_rss_conf.rss_key = NULL;
+ bond_rss_conf.rss_hf = rss_hf;
+
+ retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id,
+ &bond_rss_conf);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set slaves hash function");
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ retval = rte_eth_dev_rss_hash_conf_get(port->port_id,
+ &port->rss_conf);
+ TEST_ASSERT_SUCCESS(retval,
+ "Cannot take slaves RSS configuration");
+
+ TEST_ASSERT(port->rss_conf.rss_hf == rss_hf,
+ "Hash function not propagated for slave %d",
+ port->port_id);
+ }
+
+ default_rss_hf = rss_hf;
+ }
+
+ }
+
+ /*
+ * Test key propagation
+ */
+ for (i = 1; i < 10; i++) {
+
+ /* Set all keys to zero */
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ memset(port->rss_conf.rss_key, 0, 40);
+ retval = rte_eth_dev_rss_hash_update(port->port_id,
+ &port->rss_conf);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RSS keys");
+ }
+
+ memset(bond_rss_key, i, sizeof(bond_rss_key));
+ bond_rss_conf.rss_hf = default_rss_hf,
+ bond_rss_conf.rss_key = bond_rss_key;
+ bond_rss_conf.rss_key_len = 40;
+
+ retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id,
+ &bond_rss_conf);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set bonded port RSS keys");
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ retval = rte_eth_dev_rss_hash_conf_get(port->port_id,
+ &(port->rss_conf));
+
+ TEST_ASSERT_SUCCESS(retval,
+ "Cannot take slaves RSS configuration");
+
+ /* compare keys */
+ retval = memcmp(port->rss_conf.rss_key, bond_rss_key,
+ sizeof(bond_rss_key));
+ TEST_ASSERT(retval == 0, "Key value not propagated for slave %d",
+ port->port_id);
+ }
+ }
+
+ /*
+ * Test RETA propagation
+ */
+ for (i = 0; i < RXTX_QUEUE_COUNT; i++) {
+
+ /* Set all keys to zero */
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ retval = reta_set(port->port_id, (i + 1) % RXTX_QUEUE_COUNT,
+ port->dev_info.reta_size);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RETA");
+ }
+
+ TEST_ASSERT_SUCCESS(reta_set(test_params.bond_port_id,
+ i % RXTX_QUEUE_COUNT, test_params.bond_dev_info.reta_size),
+ "Cannot set bonded port RETA");
+
+ bond_reta_fetch();
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ slave_reta_fetch(port);
+ TEST_ASSERT(reta_check_synced(port) == 1, "RETAs inconsistent");
+ }
+ }
+
+ return TEST_SUCCESS;
+}
+
+/**
+ * Test propagation logic, when RX_RSS mq_mode is turned on for bonding port
+ */
+static int
+test_rss(void)
+{
+ /**
+ * Configure bonding port in RSS mq mode
+ */
+ TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id,
+ &rss_pmd_conf, 0), "Failed to configure bonding device\n");
+
+ rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info);
+
+ TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed");
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id),
+ "Failed to start bonding port (%d).", test_params.bond_port_id);
+
+ TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed");
+
+ TEST_ASSERT(slave_remove_and_add() == 1, "New slave should be synced");
+
+ remove_slaves_and_stop_bonded_device();
+
+ return TEST_SUCCESS;
+}
+
+/**
+ * Test propagation logic, when RX_RSS mq_mode is turned off for bonding port
+ */
+static int
+test_rss_lazy(void)
+{
+ TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id,
+ &default_pmd_conf, 0), "Failed to configure bonding device\n");
+
+ rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info);
+
+ TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed");
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id),
+ "Failed to start bonding port (%d).", test_params.bond_port_id);
+
+ TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed");
+
+ TEST_ASSERT(slave_remove_and_add() == 0, "New slave shouldn't be synced");
+
+ remove_slaves_and_stop_bonded_device();
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_setup(void)
+{
+ unsigned n;
+ int retval;
+ int port_id;
+ char name[256];
+ struct slave_conf *port;
+
+ if (test_params.mbuf_pool == NULL) {
+
+ test_params.mbuf_pool = rte_mempool_create("RSS_MBUF_POOL", NUM_MBUFS *
+ SLAVE_COUNT, MBUF_SIZE, MBUF_CACHE_SIZE,
+ sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init,
+ NULL, rte_pktmbuf_init, NULL, rte_socket_id(), 0);
+
+ TEST_ASSERT(test_params.mbuf_pool != NULL,
+ "rte_mempool_create failed\n");
+ }
+
+ /* Create / initialize ring eth devs. */
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ port_id = rte_eth_dev_count();
+ snprintf(name, sizeof(name), SLAVE_DEV_NAME_FMT, port_id);
+
+ retval = eth_dev_null_create(name, 0, 64, 0);
+ TEST_ASSERT_SUCCESS(retval, "Failed to create null device '%s'\n",
+ name);
+
+ port->port_id = port_id;
+
+ port->rss_conf.rss_key = port->rss_key;
+ port->rss_conf.rss_key_len = 40;
+
+ retval = configure_ethdev(port->port_id, &default_pmd_conf, 0);
+ TEST_ASSERT_SUCCESS(retval, "Failed to configure virtual ethdev %s\n",
+ name);
+
+ rte_eth_dev_info_get(port->port_id, &port->dev_info);
+ }
+
+ if (test_params.bond_port_id == INVALID_PORT_ID) {
+ retval = rte_eth_bond_create(BONDED_DEV_NAME, 0, rte_socket_id());
+
+ TEST_ASSERT(retval >= 0, "Failed to create bonded ethdev %s",
+ BONDED_DEV_NAME);
+
+ test_params.bond_port_id = retval;
+
+ TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id,
+ &default_pmd_conf, 0), "Failed to configure bonding device\n");
+
+ rte_eth_dev_info_get(test_params.bond_port_id,
+ &test_params.bond_dev_info);
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+testsuite_teardown(void)
+{
+ struct slave_conf *port;
+ uint8_t i;
+
+ /* Only stop ports.
+ * Any cleanup/reset state is done when particular test is
+ * started. */
+
+ rte_eth_dev_stop(test_params.bond_port_id);
+
+ FOR_EACH_PORT(i, port)
+ rte_eth_dev_stop(port->port_id);
+
+ return 0;
+}
+
+static int
+check_environment(void)
+{
+ return TEST_SUCCESS;
+}
+
+static int
+test_rssconf_executor(int (*test_func)(void))
+{
+ int test_result;
+
+ /* Check if environment is clean. Fail to launch a test if there was
+ * a critical error before that prevented to reset environment. */
+ TEST_ASSERT_SUCCESS(check_environment(),
+ "Refusing to launch test in dirty environment.");
+
+ RTE_VERIFY(test_func != NULL);
+ test_result = (*test_func)();
+
+ /* If test succeed check if environment wast left in good condition. */
+ if (test_result == TEST_SUCCESS)
+ test_result = check_environment();
+
+ /* Reset environment in case test failed to do that. */
+ if (test_result != TEST_SUCCESS) {
+ TEST_ASSERT_SUCCESS(remove_slaves_and_stop_bonded_device(),
+ "Failed to stop bonded device");
+ }
+
+ return test_result;
+}
+
+static int
+test_setup_wrapper(void)
+{
+ return test_rssconf_executor(&test_setup);
+}
+
+static int
+test_rss_wrapper(void)
+{
+ return test_rssconf_executor(&test_rss);
+}
+
+static int
+test_rss_lazy_wrapper(void)
+{
+ return test_rssconf_executor(&test_rss_lazy);
+}
+
+static struct unit_test_suite link_bonding_rssconf_test_suite = {
+ .suite_name = "RSS Dynamic Configuration for Bonding Unit Test Suite",
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+ TEST_CASE_NAMED("test_setup", test_setup_wrapper),
+ TEST_CASE_NAMED("test_rss", test_rss_wrapper),
+ TEST_CASE_NAMED("test_rss_lazy", test_rss_lazy_wrapper),
+ { NULL, NULL, NULL, NULL, NULL } /**< NULL terminate unit test array */
+ }
+};
+
+static int
+test_link_bonding_rssconf(void)
+{
+ return unit_test_suite_runner(&link_bonding_rssconf_test_suite);
+}
+
+static struct test_command test_link_bonding_rssconf_cmd = {
+ .command = "link_bonding_rssconf_autotest",
+ .callback = test_link_bonding_rssconf,
+};
+
+REGISTER_TEST_COMMAND(test_link_bonding_rssconf_cmd);
--
1.7.9.5
Tomasz Kulasek
2015-07-15 17:26:27 UTC
Permalink
This patch adds propagation of mapping over the slaves, and fills bonding
port's stats with a sum of corresponding values taken from bonded slaves,
when stats are requested for bonding port.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/bonding/rte_eth_bond_pmd.c | 34 +++++++++++++++++++++++++++++++-
1 file changed, 33 insertions(+), 1 deletion(-)

diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index cd23f42..ff92011 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -1783,7 +1783,7 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
{
struct bond_dev_private *internals = dev->data->dev_private;
struct rte_eth_stats slave_stats;
- int i;
+ int i, j;

for (i = 0; i < internals->slave_count; i++) {
rte_eth_stats_get(internals->slaves[i].port_id, &slave_stats);
@@ -1802,6 +1802,15 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
stats->rx_pause_xon += slave_stats.rx_pause_xon;
stats->tx_pause_xoff += slave_stats.tx_pause_xoff;
stats->rx_pause_xoff += slave_stats.rx_pause_xoff;
+
+ for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) {
+ stats->q_ipackets[j] += slave_stats.q_ipackets[j];
+ stats->q_opackets[j] += slave_stats.q_opackets[j];
+ stats->q_ibytes[j] += slave_stats.q_ibytes[j];
+ stats->q_obytes[j] += slave_stats.q_obytes[j];
+ stats->q_errors[j] += slave_stats.q_errors[j];
+ }
+
}
}

@@ -2110,6 +2119,28 @@ bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev,
return 0;
}

+static int
+bond_ethdev_queue_stats_mapping_set(struct rte_eth_dev *dev, uint16_t queue_id,
+ uint8_t stat_idx, uint8_t is_rx)
+{
+ int i;
+ int retval;
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ for (i = 0; i < internals->slave_count; i++) {
+ if (is_rx)
+ retval = rte_eth_dev_set_rx_queue_stats_mapping(
+ internals->slaves[i].port_id, queue_id, stat_idx);
+ else
+ retval = rte_eth_dev_set_tx_queue_stats_mapping(
+ internals->slaves[i].port_id, queue_id, stat_idx);
+ if (retval != 0)
+ return retval;
+ }
+
+ return 0;
+}
+
struct eth_dev_ops default_dev_ops = {
.dev_start = bond_ethdev_start,
.dev_stop = bond_ethdev_stop,
@@ -2123,6 +2154,7 @@ struct eth_dev_ops default_dev_ops = {
.link_update = bond_ethdev_link_update,
.stats_get = bond_ethdev_stats_get,
.stats_reset = bond_ethdev_stats_reset,
+ .queue_stats_mapping_set = bond_ethdev_queue_stats_mapping_set,
.promiscuous_enable = bond_ethdev_promiscuous_enable,
.promiscuous_disable = bond_ethdev_promiscuous_disable,
.reta_update = bond_ethdev_rss_reta_update,
--
1.7.9.5
Thomas Monjalon
2015-09-25 10:14:42 UTC
Permalink
Post by Tomasz Kulasek
+ .queue_stats_mapping_set = bond_ethdev_queue_stats_mapping_set,
As explained with previous version of this patchset, this API should be
removed. It is specific to some Intel devices.
Please do not use it in bonding.
Then we could discuss in another thread how to remove it from ethdev.
Kulasek, TomaszX
2015-09-25 13:26:05 UTC
Permalink
-----Original Message-----
Sent: Friday, September 25, 2015 12:15
To: Kulasek, TomaszX
Subject: Re: [dpdk-dev] [PATCHv4 7/9] bonding: queue stats mapping
Post by Tomasz Kulasek
+ .queue_stats_mapping_set =
bond_ethdev_queue_stats_mapping_set,
As explained with previous version of this patchset, this API should be
removed. It is specific to some Intel devices.
Please do not use it in bonding.
Then we could discuss in another thread how to remove it from ethdev.
Ok, I remove it in v5.
Tomasz Kulasek
2015-07-15 17:26:28 UTC
Permalink
Signed-off-by: Tomasz Kulasek <***@intel.com>
---
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
index 96e554f..03baf90 100644
--- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
+++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
@@ -188,7 +188,7 @@ conditions are not met. If a user wishes to monitor individual slaves then they
must register callbacks with that slave directly.

The link bonding library also supports devices which do not implement link
-status change interrupts, this is achieve by polling the devices link status at
+status change interrupts, this is achieved by polling the devices link status at
a defined period which is set using the ``rte_eth_bond_link_monitoring_set``
API, the default polling interval is 10ms. When a device is added as a slave to
a bonding device it is determined using the ``RTE_PCI_DRV_INTR_LSC`` flag
@@ -286,7 +286,7 @@ and UDP protocols for load balancing.
Using Link Bonding Devices
--------------------------

-The librte_pmd_bond library support two modes of device creation, the libraries
+The librte_pmd_bond library supports two modes of device creation, the libraries
export full C API or using the EAL command line to statically configure link
bonding devices at application startup. Using the EAL option it is possible to
use link bonding functionality transparently without specific knowledge of the
@@ -299,7 +299,7 @@ Using the Poll Mode Driver from an Application

Using the librte_pmd_bond libraries API it is possible to dynamically create
and manage link bonding device from within any application. Link bonding
-device are created using the ``rte_eth_bond_create`` API which requires a
+devices are created using the ``rte_eth_bond_create`` API which requires a
unique device name, the link bonding mode to initial the device in and finally
the socket Id which to allocate the devices resources onto. After successful
creation of a bonding device it must be configured using the generic Ethernet
@@ -362,7 +362,7 @@ The different options are:
mode=2

* slave: Defines the PMD device which will be added as slave to the bonded
- device. This option can be selected multiple time, for each device to be
+ device. This option can be selected multiple times, for each device to be
added as a slave. Physical devices should be specified using their PCI
address, in the format domain:bus:devid.function
--
1.7.9.5
Tomasz Kulasek
2015-07-15 17:26:29 UTC
Permalink
Documentation update about implementation details and requirements for
Dynamic RSS Configuration for Bonding.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 34 ++++++++++++++++++--
1 file changed, 32 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
index 03baf90..46f0296 100644
--- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
+++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
@@ -1,5 +1,5 @@
.. BSD LICENSE
- Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+ Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
All rights reserved.

Redistribution and use in source and binary forms, with or without
@@ -173,7 +173,28 @@ After a slave device is added to a bonded device slave is stopped using
``rte_eth_dev_stop`` and then reconfigured using ``rte_eth_dev_configure``
the RX and TX queues are also reconfigured using ``rte_eth_tx_queue_setup`` /
``rte_eth_rx_queue_setup`` with the parameters use to configure the bonding
-device.
+device. If RSS is enabled for bonding device, this mode is also enabled on new
+slave and configured as well.
+
+Setting up multi-queue mode for bonding device to RSS, makes it fully
+RSS-capable, so all slaves are synchronized with its configuration. This mode is
+intended to provide RSS configuration on slaves transparent for client
+application implementation.
+
+Bonding device stores its own version of RSS settings i.e. RETA, RSS hash
+function and RSS key, used to set up its slaves. That let to define the meaning
+of RSS configuration of bonding device as desired configuration of whole bonding
+(as one unit), without pointing any of slave inside. It is required to ensure
+consistency and made it more errorproof.
+
+RSS hash function set for bonding device, is a maximal set of RSS hash functions
+supported by all bonded slaves. RETA size is a GCD of all its RETA's sizes, so
+it can be easily used as a pattern providing expected behavior, even if slave
+RETAs' sizes are different. If RSS Key is not set for bonded device, it's not
+changed on the slaves and default key for device is used.
+
+All settings are managed through the bonding port API and always are propagated
+in one direction (from bonding to slaves).

Link Status Change Interrupts / Polling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -207,6 +228,15 @@ these parameters.
A bonding device must have a minimum of one slave before the bonding device
itself can be started.

+To use a bonding device dynamic RSS configuration feature effectively, it is
+also required, that all slaves should be RSS-capable and support, at least one
+common hash function available for each of them. Changing RSS key is only
+possible, when all slave devices support the same key size.
+
+To prevent inconsistency on how slaves process packets, once a device is added
+to a bonding device, RSS configuration should be managed through the bonding
+device API, and not directly on the slave.
+
Like all other PMD, all functions exported by a PMD are lock-free functions
that are assumed not to be invoked in parallel on different logical cores to
work on the same target object.
--
1.7.9.5
Thomas Monjalon
2015-09-25 10:16:52 UTC
Permalink
Post by Tomasz Kulasek
bonding: rss dynamic configuration
null: fix segfault when null_pmd added to bonding
null: extend number of virtual queues
null: virtual dynamic rss configuration
null: export eth_dev_null_create
Please Tetsuya, could you review the patches on null PMD?
Thanks
Kulasek, TomaszX
2015-09-25 13:28:52 UTC
Permalink
-----Original Message-----
Sent: Friday, September 25, 2015 12:17
To: Tetsuya Mukawa
Subject: Re: [dpdk-dev] [PATCHv4 0/9] Dynamic RSS Configuration for
Bonding
Post by Tomasz Kulasek
bonding: rss dynamic configuration
null: fix segfault when null_pmd added to bonding
null: extend number of virtual queues
null: virtual dynamic rss configuration
null: export eth_dev_null_create
Please Tetsuya, could you review the patches on null PMD?
Thanks
Do you need patch-set v5 or should I wait?

Tomasz
Thomas Monjalon
2015-09-25 13:44:38 UTC
Permalink
Post by Kulasek, TomaszX
Post by Thomas Monjalon
Please Tetsuya, could you review the patches on null PMD?
Thanks
Do you need patch-set v5 or should I wait?
I think you should wait for Tetsuya feedback.
Tomasz Kulasek
2015-09-30 14:04:56 UTC
Permalink
OVERVIEW
--------
1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding
device fully RSS-capable, so all slaves are synchronized with its configuration.
This mode is intended to provide RSS configuration as known from "dynamic RSS
configuration for one port" and made slaves transparent for client application
implementation.

2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not
synchronized. That provides an ability to configure them manually. This mode may
be useful when application wants to manage RSS in an unusual way and the
consistency of RSS configuration for slaves isn't required.

Turning on/off RSS mode for slaves when bonding is started is not possible.
Other RSS configuration is propagated over slaves, when bonding device API is
used to do it.

v5 changes:
- updated to DPDK 2.2
- removed copyright change from null device source
- removed queue_stats_mapping_set from eth_dev_ops of bonding device
- null pmd cleanups (removed unnecessary malloc, replaced memcpy with
rte_memcpy)
- fixed queues number configuration in null pmd

v4 changes:
- fixed copy-paste error,
- removed example application as too complex and introducing a new
dependency,
- addapted null pmd to be used as testing device for dynamic RSS configuration,
- addapted test units to use null pmd instead of ring pmd,
- ring pmd is not used and changed in this patchset

v3 changes:
- checkpatch cleanups

v2 changes:
- added support for keys other than 40 bytes long,
- now, if RSS key is not set for bonding, it is not set also for slaves,
- fix - full initial RSS configuration before any slave is added was not
possible due to the initially zeroed flow_type_rss_offloads for bonding,
- fix - changed error to warning when slave is synchronizing due to the
bonding's initial configuration (to allow use slaves' drivers not supporting
dynamic RSS configuration in bonding),
- some code cleanups,
- updated documentation,

Tomasz Kulasek (9):
bonding: rss dynamic configuration
null: fix segfault when null_pmd added to bonding
null: extend number of virtual queues
null: virtual dynamic rss configuration
null: export eth_dev_null_create
test: dynamic rss configuration
bonding: per queue stats
doc: fixed spellings and typos
doc: dynamic rss configuration for bonding

app/test/Makefile | 8 +
app/test/test_link_bonding_rssconf.c | 679 ++++++++++++++++++++
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 42 +-
drivers/net/bonding/rte_eth_bond_api.c | 28 +
drivers/net/bonding/rte_eth_bond_pmd.c | 216 ++++++-
drivers/net/bonding/rte_eth_bond_private.h | 12 +
drivers/net/null/Makefile | 2 +-
drivers/net/null/rte_eth_null.c | 148 ++++-
drivers/net/null/rte_eth_null.h | 40 ++
drivers/net/null/rte_pmd_null_version.map | 7 +
10 files changed, 1150 insertions(+), 32 deletions(-)
create mode 100644 app/test/test_link_bonding_rssconf.c
create mode 100644 drivers/net/null/rte_eth_null.h
--
1.7.9.5
Tomasz Kulasek
2015-09-30 14:04:57 UTC
Permalink
Bonding device implements independent management of RSS settings. It
stores its own copies of settings i.e. RETA, RSS hash function and RSS
key. It’s required to ensure consistency.

1) RSS hash function set for bonding device is maximal set of RSS hash
functions supported by all bonded devices. That mean, to have RSS support
for bonding, all slaves should be RSS-capable.

2) RSS key is propagated over the slaves "as is".

3) RETA for bonding is an internal table managed by bonding API, and is
used as a pattern to set up slaves. Its size is GCD of all RETA sizes, so
it can be easily used as a pattern providing expected behavior, even if
slaves RETA sizes are different.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/bonding/rte_eth_bond_api.c | 28 ++++
drivers/net/bonding/rte_eth_bond_pmd.c | 205 ++++++++++++++++++++++++++--
drivers/net/bonding/rte_eth_bond_private.h | 12 ++
3 files changed, 231 insertions(+), 14 deletions(-)

diff --git a/drivers/net/bonding/rte_eth_bond_api.c b/drivers/net/bonding/rte_eth_bond_api.c
index 0681d1a..92073df 100644
--- a/drivers/net/bonding/rte_eth_bond_api.c
+++ b/drivers/net/bonding/rte_eth_bond_api.c
@@ -273,6 +273,9 @@ rte_eth_bond_create(const char *name, uint8_t mode, uint8_t socket_id)
internals->rx_offload_capa = 0;
internals->tx_offload_capa = 0;

+ /* Initially allow to choose any offload type */
+ internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK;
+
memset(internals->active_slaves, 0, sizeof(internals->active_slaves));
memset(internals->slaves, 0, sizeof(internals->slaves));

@@ -369,6 +372,11 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)

rte_eth_dev_info_get(slave_port_id, &dev_info);

+ /* We need to store slaves reta_size to be able to synchronize RETA for all
+ * slave devices even if its sizes are different.
+ */
+ internals->slaves[internals->slave_count].reta_size = dev_info.reta_size;
+
if (internals->slave_count < 1) {
/* if MAC is not user defined then use MAC of first slave add to
* bonded device */
@@ -382,9 +390,16 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
/* Make primary slave */
internals->primary_port = slave_port_id;

+ /* Inherit queues settings from first slave */
+ internals->nb_rx_queues = slave_eth_dev->data->nb_rx_queues;
+ internals->nb_tx_queues = slave_eth_dev->data->nb_tx_queues;
+
+ internals->reta_size = dev_info.reta_size;
+
/* Take the first dev's offload capabilities */
internals->rx_offload_capa = dev_info.rx_offload_capa;
internals->tx_offload_capa = dev_info.tx_offload_capa;
+ internals->flow_type_rss_offloads = dev_info.flow_type_rss_offloads;

} else {
/* Check slave link properties are supported if props are set,
@@ -403,8 +418,19 @@ __eth_bond_slave_add_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
}
internals->rx_offload_capa &= dev_info.rx_offload_capa;
internals->tx_offload_capa &= dev_info.tx_offload_capa;
+ internals->flow_type_rss_offloads &= dev_info.flow_type_rss_offloads;
+
+ /* RETA size is GCD of all slaves RETA sizes, so, if all sizes will be
+ * the power of 2, the lower one is GCD
+ */
+ if (internals->reta_size > dev_info.reta_size)
+ internals->reta_size = dev_info.reta_size;
+
}

+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf &=
+ internals->flow_type_rss_offloads;
+
internals->slave_count++;

/* Update all slave devices MACs*/
@@ -531,6 +557,8 @@ __eth_bond_slave_remove_lock_free(uint8_t bonded_port_id, uint8_t slave_port_id)
if (internals->slave_count == 0) {
internals->rx_offload_capa = 0;
internals->tx_offload_capa = 0;
+ internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK;
+ internals->reta_size = 0;
}
return 0;
}
diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 5cc6372..2880f5c 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -1310,6 +1310,23 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev,
if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC)
slave_eth_dev->data->dev_conf.intr_conf.lsc = 1;

+ /* If RSS is enabled for bonding, try to enable it for slaves */
+ if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) {
+ if (bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len
+ != 0) {
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len =
+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len;
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key =
+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key;
+ } else {
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = NULL;
+ }
+
+ slave_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ bonded_eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ slave_eth_dev->data->dev_conf.rxmode.mq_mode |= ETH_MQ_RX_RSS;
+ }
+
/* Configure device */
errval = rte_eth_dev_configure(slave_eth_dev->data->port_id,
bonded_eth_dev->data->nb_rx_queues,
@@ -1361,6 +1378,30 @@ slave_configure(struct rte_eth_dev *bonded_eth_dev,
return -1;
}

+ /* If RSS is enabled for bonding, synchronize RETA */
+ if (bonded_eth_dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) {
+ int i;
+ struct bond_dev_private *internals;
+
+ internals = bonded_eth_dev->data->dev_private;
+
+ for (i = 0; i < internals->slave_count; i++) {
+ if (internals->slaves[i].port_id == slave_eth_dev->data->port_id) {
+ errval = rte_eth_dev_rss_reta_update(
+ slave_eth_dev->data->port_id,
+ &internals->reta_conf[0],
+ internals->slaves[i].reta_size);
+ if (errval != 0) {
+ RTE_LOG(WARNING, PMD,
+ "rte_eth_dev_rss_reta_update on slave port %d fails (err %d)."
+ " RSS Configuration for bonding may be inconsistent.\n",
+ slave_eth_dev->data->port_id, errval);
+ }
+ break;
+ }
+ }
+ }
+
/* If lsc interrupt is set, check initial slave's link status */
if (slave_eth_dev->driver->pci_drv.drv_flags & RTE_PCI_DRV_INTR_LSC)
bond_ethdev_lsc_event_callback(slave_eth_dev->data->port_id,
@@ -1596,6 +1637,9 @@ bond_ethdev_info(struct rte_eth_dev *dev, struct rte_eth_dev_info *dev_info)

dev_info->rx_offload_capa = internals->rx_offload_capa;
dev_info->tx_offload_capa = internals->tx_offload_capa;
+ dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads;
+
+ dev_info->reta_size = internals->reta_size;
}

static int
@@ -1977,21 +2021,132 @@ bond_ethdev_lsc_event_callback(uint8_t port_id, enum rte_eth_event_type type,
}
}

+static int
+bond_ethdev_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ unsigned i, j;
+ int result = 0;
+ int slave_reta_size;
+ unsigned reta_count;
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ if (reta_size != internals->reta_size)
+ return -EINVAL;
+
+ /* Copy RETA table */
+ reta_count = reta_size / RTE_RETA_GROUP_SIZE;
+
+ for (i = 0; i < reta_count; i++) {
+ internals->reta_conf[i].mask = reta_conf[i].mask;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ internals->reta_conf[i].reta[j] = reta_conf[i].reta[j];
+ }
+
+ /* Fill rest of array */
+ for (; i < RTE_DIM(internals->reta_conf); i += reta_count)
+ memcpy(&internals->reta_conf[i], &internals->reta_conf[0],
+ sizeof(internals->reta_conf[0]) * reta_count);
+
+ /* Propagate RETA over slaves */
+ for (i = 0; i < internals->slave_count; i++) {
+ slave_reta_size = internals->slaves[i].reta_size;
+ result = rte_eth_dev_rss_reta_update(internals->slaves[i].port_id,
+ &internals->reta_conf[0], slave_reta_size);
+ if (result < 0)
+ return result;
+ }
+
+ return 0;
+}
+
+static int
+bond_ethdev_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ int i, j;
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ if (reta_size != internals->reta_size)
+ return -EINVAL;
+
+ /* Copy RETA table */
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++)
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ reta_conf[i].reta[j] = internals->reta_conf[i].reta[j];
+
+ return 0;
+}
+
+static int
+bond_ethdev_rss_hash_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ int i, result = 0;
+ struct bond_dev_private *internals = dev->data->dev_private;
+ struct rte_eth_rss_conf bond_rss_conf;
+
+ memcpy(&bond_rss_conf, rss_conf, sizeof(struct rte_eth_rss_conf));
+
+ bond_rss_conf.rss_hf &= internals->flow_type_rss_offloads;
+
+ if (bond_rss_conf.rss_hf != 0)
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf = bond_rss_conf.rss_hf;
+
+ if (bond_rss_conf.rss_key && bond_rss_conf.rss_key_len <
+ sizeof(internals->rss_key)) {
+ if (bond_rss_conf.rss_key_len == 0)
+ bond_rss_conf.rss_key_len = 40;
+ internals->rss_key_len = bond_rss_conf.rss_key_len;
+ memcpy(internals->rss_key, bond_rss_conf.rss_key,
+ internals->rss_key_len);
+ }
+
+ for (i = 0; i < internals->slave_count; i++) {
+ result = rte_eth_dev_rss_hash_update(internals->slaves[i].port_id,
+ &bond_rss_conf);
+ if (result < 0)
+ return result;
+ }
+
+ return 0;
+}
+
+static int
+bond_ethdev_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct bond_dev_private *internals = dev->data->dev_private;
+
+ rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ rss_conf->rss_key_len = internals->rss_key_len;
+ if (rss_conf->rss_key)
+ memcpy(rss_conf->rss_key, internals->rss_key, internals->rss_key_len);
+
+ return 0;
+}
+
struct eth_dev_ops default_dev_ops = {
- .dev_start = bond_ethdev_start,
- .dev_stop = bond_ethdev_stop,
- .dev_close = bond_ethdev_close,
- .dev_configure = bond_ethdev_configure,
- .dev_infos_get = bond_ethdev_info,
- .rx_queue_setup = bond_ethdev_rx_queue_setup,
- .tx_queue_setup = bond_ethdev_tx_queue_setup,
- .rx_queue_release = bond_ethdev_rx_queue_release,
- .tx_queue_release = bond_ethdev_tx_queue_release,
- .link_update = bond_ethdev_link_update,
- .stats_get = bond_ethdev_stats_get,
- .stats_reset = bond_ethdev_stats_reset,
- .promiscuous_enable = bond_ethdev_promiscuous_enable,
- .promiscuous_disable = bond_ethdev_promiscuous_disable
+ .dev_start = bond_ethdev_start,
+ .dev_stop = bond_ethdev_stop,
+ .dev_close = bond_ethdev_close,
+ .dev_configure = bond_ethdev_configure,
+ .dev_infos_get = bond_ethdev_info,
+ .rx_queue_setup = bond_ethdev_rx_queue_setup,
+ .tx_queue_setup = bond_ethdev_tx_queue_setup,
+ .rx_queue_release = bond_ethdev_rx_queue_release,
+ .tx_queue_release = bond_ethdev_tx_queue_release,
+ .link_update = bond_ethdev_link_update,
+ .stats_get = bond_ethdev_stats_get,
+ .stats_reset = bond_ethdev_stats_reset,
+ .promiscuous_enable = bond_ethdev_promiscuous_enable,
+ .promiscuous_disable = bond_ethdev_promiscuous_disable,
+ .reta_update = bond_ethdev_rss_reta_update,
+ .reta_query = bond_ethdev_rss_reta_query,
+ .rss_hash_update = bond_ethdev_rss_hash_update,
+ .rss_hash_conf_get = bond_ethdev_rss_hash_conf_get
};

static int
@@ -2090,6 +2245,28 @@ bond_ethdev_configure(struct rte_eth_dev *dev)
int arg_count;
uint8_t port_id = dev - rte_eth_devices;

+ static const uint8_t default_rss_key[40] = {
+ 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D,
+ 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+ 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B,
+ 0xBE, 0xAC, 0x01, 0xFA
+ };
+
+ unsigned i, j;
+
+ /* If RSS is enabled, fill table and key with default values */
+ if (dev->data->dev_conf.rxmode.mq_mode & ETH_MQ_RX_RSS) {
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key = internals->rss_key;
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_key_len = 0;
+ memcpy(internals->rss_key, default_rss_key, 40);
+
+ for (i = 0; i < RTE_DIM(internals->reta_conf); i++) {
+ internals->reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ internals->reta_conf[i].reta[j] = j % dev->data->nb_rx_queues;
+ }
+ }
+
/*
* if no kvlist, it means that this bonded device has been created
* through the bonding api.
diff --git a/drivers/net/bonding/rte_eth_bond_private.h b/drivers/net/bonding/rte_eth_bond_private.h
index 038bca6..e7af809 100644
--- a/drivers/net/bonding/rte_eth_bond_private.h
+++ b/drivers/net/bonding/rte_eth_bond_private.h
@@ -103,6 +103,8 @@ struct bond_slave_details {
uint8_t last_link_status;
/**< Port Id of slave eth_dev */
struct ether_addr persisted_mac_addr;
+
+ uint16_t reta_size;
};


@@ -155,6 +157,16 @@ struct bond_dev_private {
uint32_t rx_offload_capa; /** Rx offload capability */
uint32_t tx_offload_capa; /** Tx offload capability */

+ /** Bit mask of RSS offloads, the bit offset also means flow type */
+ uint64_t flow_type_rss_offloads;
+
+ uint16_t reta_size;
+ struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_512 /
+ RTE_RETA_GROUP_SIZE];
+
+ uint8_t rss_key[52]; /**< 52-byte hash key buffer. */
+ uint8_t rss_key_len; /**< hash key length in bytes. */
+
struct rte_kvargs *kvlist;
uint8_t slave_update_idx;
};
--
1.7.9.5
Tomasz Kulasek
2015-09-30 14:04:58 UTC
Permalink
This patch initializes eth_dev->link_intr_cbs queue used when null pmd is
added to the bonding.

v5 changes:
- removed unnecessary malloc for eth_driver (rte_null_pmd)

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/null/rte_eth_null.c | 2 ++
1 file changed, 2 insertions(+)

diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index e244595..c748101 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -432,6 +432,7 @@ eth_dev_null_create(const char *name,
internals->numa_node = numa_node;

pci_dev->numa_node = numa_node;
+ pci_dev->driver = &rte_null_pmd.pci_drv;

data->dev_private = internals;
data->port_id = eth_dev->data->port_id;
@@ -445,6 +446,7 @@ eth_dev_null_create(const char *name,
eth_dev->dev_ops = &ops;
eth_dev->pci_dev = pci_dev;
eth_dev->driver = &rte_null_pmd;
+ TAILQ_INIT(&eth_dev->link_intr_cbs);

/* finally assign rx and tx ops */
if (packet_copy) {
--
1.7.9.5
Tetsuya Mukawa
2015-10-14 01:34:30 UTC
Permalink
Post by Tomasz Kulasek
This patch initializes eth_dev->link_intr_cbs queue used when null pmd is
added to the bonding.
- removed unnecessary malloc for eth_driver (rte_null_pmd)
---
drivers/net/null/rte_eth_null.c | 2 ++
1 file changed, 2 insertions(+)
Acked-by: Tetsuya Mukawa <***@igel.co.jp>
Tomasz Kulasek
2015-09-30 14:04:59 UTC
Permalink
This patch adds a possibility to configure more than one queue on null
device.

v5 changes:
- fixed queues number configuration (using internals->nb_*_queues instead
of dev->data->nb_*_queues)

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/null/rte_eth_null.c | 28 +++++++++++++++++++---------
1 file changed, 19 insertions(+), 9 deletions(-)

diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index c748101..bf81b1b 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -71,8 +71,8 @@ struct pmd_internals {
unsigned nb_rx_queues;
unsigned nb_tx_queues;

- struct null_queue rx_null_queues[1];
- struct null_queue tx_null_queues[1];
+ struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT];
+ struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT];
};


@@ -178,7 +178,15 @@ eth_null_copy_tx(void *q, struct rte_mbuf **bufs, uint16_t nb_bufs)
}

static int
-eth_dev_configure(struct rte_eth_dev *dev __rte_unused) { return 0; }
+eth_dev_configure(struct rte_eth_dev *dev) {
+ struct pmd_internals *internals;
+
+ internals = dev->data->dev_private;
+ internals->nb_rx_queues = dev->data->nb_rx_queues;
+ internals->nb_tx_queues = dev->data->nb_tx_queues;
+
+ return 0;
+}

static int
eth_dev_start(struct rte_eth_dev *dev)
@@ -213,10 +221,11 @@ eth_rx_queue_setup(struct rte_eth_dev *dev, uint16_t rx_queue_id,
if ((dev == NULL) || (mb_pool == NULL))
return -EINVAL;

- if (rx_queue_id != 0)
+ internals = dev->data->dev_private;
+
+ if (rx_queue_id >= internals->nb_rx_queues)
return -ENODEV;

- internals = dev->data->dev_private;
packet_size = internals->packet_size;

internals->rx_null_queues[rx_queue_id].mb_pool = mb_pool;
@@ -246,10 +255,11 @@ eth_tx_queue_setup(struct rte_eth_dev *dev, uint16_t tx_queue_id,
if (dev == NULL)
return -EINVAL;

- if (tx_queue_id != 0)
+ internals = dev->data->dev_private;
+
+ if (tx_queue_id >= internals->nb_tx_queues)
return -ENODEV;

- internals = dev->data->dev_private;
packet_size = internals->packet_size;

dev->data->tx_queues[tx_queue_id] =
@@ -279,8 +289,8 @@ eth_dev_info(struct rte_eth_dev *dev,
dev_info->driver_name = drivername;
dev_info->max_mac_addrs = 1;
dev_info->max_rx_pktlen = (uint32_t)-1;
- dev_info->max_rx_queues = (uint16_t)internals->nb_rx_queues;
- dev_info->max_tx_queues = (uint16_t)internals->nb_tx_queues;
+ dev_info->max_rx_queues = RTE_DIM(internals->rx_null_queues);
+ dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues);
dev_info->min_rx_bufsize = 0;
dev_info->pci_dev = NULL;
}
--
1.7.9.5
Tetsuya Mukawa
2015-10-14 01:34:46 UTC
Permalink
Post by Tomasz Kulasek
This patch adds a possibility to configure more than one queue on null
device.
- fixed queues number configuration (using internals->nb_*_queues instead
of dev->data->nb_*_queues)
---
drivers/net/null/rte_eth_null.c | 28 +++++++++++++++++++---------
1 file changed, 19 insertions(+), 9 deletions(-)
Acked-by: Tetsuya Mukawa <***@igel.co.jp>
Tomasz Kulasek
2015-09-30 14:05:00 UTC
Permalink
This implementation allows to set and read RSS configuration for null
device, and is used to validate right values propagation over the slaves,
in test units for dynamic RSS configuration for bonding.

v5 changes:
- replaced memcpy with rte_memcpy

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/null/rte_eth_null.c | 116 +++++++++++++++++++++++++++++++++++++++
1 file changed, 116 insertions(+)

diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index bf81b1b..b01f647 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -37,6 +37,8 @@
#include <rte_memcpy.h>
#include <rte_dev.h>
#include <rte_kvargs.h>
+#include <rte_eth_null.h>
+#include <rte_spinlock.h>

#define ETH_NULL_PACKET_SIZE_ARG "size"
#define ETH_NULL_PACKET_COPY_ARG "copy"
@@ -73,6 +75,17 @@ struct pmd_internals {

struct null_queue rx_null_queues[RTE_MAX_QUEUES_PER_PORT];
struct null_queue tx_null_queues[RTE_MAX_QUEUES_PER_PORT];
+
+ /** Bit mask of RSS offloads, the bit offset also means flow type */
+ uint64_t flow_type_rss_offloads;
+
+ rte_spinlock_t rss_lock;
+
+ uint16_t reta_size;
+ struct rte_eth_rss_reta_entry64 reta_conf[ETH_RSS_RETA_SIZE_128 /
+ RTE_RETA_GROUP_SIZE];
+
+ uint8_t rss_key[40]; /**< 40-byte hash key. */
};


@@ -293,6 +306,8 @@ eth_dev_info(struct rte_eth_dev *dev,
dev_info->max_tx_queues = RTE_DIM(internals->tx_null_queues);
dev_info->min_rx_bufsize = 0;
dev_info->pci_dev = NULL;
+ dev_info->reta_size = internals->reta_size;
+ dev_info->flow_type_rss_offloads = internals->flow_type_rss_offloads;
}

static void
@@ -373,6 +388,91 @@ static int
eth_link_update(struct rte_eth_dev *dev __rte_unused,
int wait_to_complete __rte_unused) { return 0; }

+static int
+eth_rss_reta_update(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ int i, j;
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ if (reta_size != internal->reta_size)
+ return -EINVAL;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ /* Copy RETA table */
+ for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) {
+ internal->reta_conf[i].mask = reta_conf[i].mask;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ internal->reta_conf[i].reta[j] = reta_conf[i].reta[j];
+ }
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_reta_query(struct rte_eth_dev *dev,
+ struct rte_eth_rss_reta_entry64 *reta_conf, uint16_t reta_size)
+{
+ int i, j;
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ if (reta_size != internal->reta_size)
+ return -EINVAL;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ /* Copy RETA table */
+ for (i = 0; i < (internal->reta_size / RTE_RETA_GROUP_SIZE); i++) {
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ if ((reta_conf[i].mask >> j) & 0x01)
+ reta_conf[i].reta[j] = internal->reta_conf[i].reta[j];
+ }
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_hash_update(struct rte_eth_dev *dev, struct rte_eth_rss_conf *rss_conf)
+{
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ if ((rss_conf->rss_hf & internal->flow_type_rss_offloads) != 0)
+ dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf =
+ rss_conf->rss_hf & internal->flow_type_rss_offloads;
+
+ if (rss_conf->rss_key)
+ rte_memcpy(internal->rss_key, rss_conf->rss_key, 40);
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
+static int
+eth_rss_hash_conf_get(struct rte_eth_dev *dev,
+ struct rte_eth_rss_conf *rss_conf)
+{
+ struct pmd_internals *internal = dev->data->dev_private;
+
+ rte_spinlock_lock(&internal->rss_lock);
+
+ rss_conf->rss_hf = dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
+ if (rss_conf->rss_key)
+ rte_memcpy(rss_conf->rss_key, internal->rss_key, 40);
+
+ rte_spinlock_unlock(&internal->rss_lock);
+
+ return 0;
+}
+
static const struct eth_dev_ops ops = {
.dev_start = eth_dev_start,
.dev_stop = eth_dev_stop,
@@ -385,6 +485,10 @@ static const struct eth_dev_ops ops = {
.link_update = eth_link_update,
.stats_get = eth_stats_get,
.stats_reset = eth_stats_reset,
+ .reta_update = eth_rss_reta_update,
+ .reta_query = eth_rss_reta_query,
+ .rss_hash_update = eth_rss_hash_update,
+ .rss_hash_conf_get = eth_rss_hash_conf_get
};

static int
@@ -400,6 +504,13 @@ eth_dev_null_create(const char *name,
struct pmd_internals *internals = NULL;
struct rte_eth_dev *eth_dev = NULL;

+ static const uint8_t default_rss_key[40] = {
+ 0x6D, 0x5A, 0x56, 0xDA, 0x25, 0x5B, 0x0E, 0xC2, 0x41, 0x67, 0x25, 0x3D,
+ 0x43, 0xA3, 0x8F, 0xB0, 0xD0, 0xCA, 0x2B, 0xCB, 0xAE, 0x7B, 0x30, 0xB4,
+ 0x77, 0xCB, 0x2D, 0xA3, 0x80, 0x30, 0xF2, 0x0C, 0x6A, 0x42, 0xB7, 0x3B,
+ 0xBE, 0xAC, 0x01, 0xFA
+ };
+
if (name == NULL)
return -EINVAL;

@@ -441,6 +552,11 @@ eth_dev_null_create(const char *name,
internals->packet_copy = packet_copy;
internals->numa_node = numa_node;

+ internals->flow_type_rss_offloads = ETH_RSS_PROTO_MASK;
+ internals->reta_size = RTE_DIM(internals->reta_conf) * RTE_RETA_GROUP_SIZE;
+
+ rte_memcpy(internals->rss_key, default_rss_key, 40);
+
pci_dev->numa_node = numa_node;
pci_dev->driver = &rte_null_pmd.pci_drv;
--
1.7.9.5
Tetsuya Mukawa
2015-10-14 01:34:57 UTC
Permalink
Post by Tomasz Kulasek
This implementation allows to set and read RSS configuration for null
device, and is used to validate right values propagation over the slaves,
in test units for dynamic RSS configuration for bonding.
- replaced memcpy with rte_memcpy
---
drivers/net/null/rte_eth_null.c | 116 +++++++++++++++++++++++++++++++++++++++
1 file changed, 116 insertions(+)
Acked-by: Tetsuya Mukawa <***@igel.co.jp>
Tetsuya Mukawa
2015-10-15 07:46:16 UTC
Permalink
Post by Tomasz Kulasek
This implementation allows to set and read RSS configuration for null
device, and is used to validate right values propagation over the slaves,
in test units for dynamic RSS configuration for bonding.
- replaced memcpy with rte_memcpy
---
drivers/net/null/rte_eth_null.c | 116 +++++++++++++++++++++++++++++++++++++++
1 file changed, 116 insertions(+)
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index bf81b1b..b01f647 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -37,6 +37,8 @@
#include <rte_memcpy.h>
#include <rte_dev.h>
#include <rte_kvargs.h>
+#include <rte_eth_null.h>
+#include <rte_spinlock.h>
Hi Tomasz,

We don't have "rte_eth_null.h" at this point.
(The header file will be added next patch)
Probably, we also need "rte_pmd_null_version.map" to compile correctly.
(To make sure, please compile DPDK with "CONFIG_RTE_BUILD_SHARED_LIB=y"
option.)

Also, it seems 'rte_eth_null.h' should be included like below.
#include "rte_eth_null.h"
Without it, we cannot compile.

Thanks,
Tetsuya
Kulasek, TomaszX
2015-10-15 08:42:23 UTC
Permalink
-----Original Message-----
Sent: Thursday, October 15, 2015 09:46
Subject: Re: [dpdk-dev] [PATCH v5 4/9] null: virtual dynamic rss
configuration
Post by Tomasz Kulasek
This implementation allows to set and read RSS configuration for null
device, and is used to validate right values propagation over the
slaves, in test units for dynamic RSS configuration for bonding.
- replaced memcpy with rte_memcpy
---
drivers/net/null/rte_eth_null.c | 116
+++++++++++++++++++++++++++++++++++++++
1 file changed, 116 insertions(+)
diff --git a/drivers/net/null/rte_eth_null.c
b/drivers/net/null/rte_eth_null.c index bf81b1b..b01f647 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -37,6 +37,8 @@
#include <rte_memcpy.h>
#include <rte_dev.h>
#include <rte_kvargs.h>
+#include <rte_eth_null.h>
+#include <rte_spinlock.h>
Hi Tomasz,
We don't have "rte_eth_null.h" at this point.
(The header file will be added next patch) Probably, we also need
"rte_pmd_null_version.map" to compile correctly.
(To make sure, please compile DPDK with "CONFIG_RTE_BUILD_SHARED_LIB=y"
option.)
Also, it seems 'rte_eth_null.h' should be included like below.
#include "rte_eth_null.h"
Without it, we cannot compile.
Thanks,
Tetsuya
Hi Tetsuya,

This file and modifications are already included in "[dpdk-dev,v5,5/9] null: export eth_dev_null_create" in the patch set, also the required symlink is created like below:

---
diff --git a/drivers/net/null/Makefile b/drivers/net/null/Makefile
index 96ba01c..2202389 100644
--- a/drivers/net/null/Makefile
+++ b/drivers/net/null/Makefile
@@ -51,7 +51,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += rte_eth_null.c
#
# Export include files
#
-SYMLINK-y-include +=
+SYMLINK-y-include += rte_eth_null.h

# this lib depends upon:
DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_mbuf
---

Tomasz.
Tetsuya Mukawa
2015-10-15 09:21:56 UTC
Permalink
Post by Kulasek, TomaszX
-----Original Message-----
Sent: Thursday, October 15, 2015 09:46
Subject: Re: [dpdk-dev] [PATCH v5 4/9] null: virtual dynamic rss
configuration
Post by Tomasz Kulasek
This implementation allows to set and read RSS configuration for null
device, and is used to validate right values propagation over the
slaves, in test units for dynamic RSS configuration for bonding.
- replaced memcpy with rte_memcpy
---
drivers/net/null/rte_eth_null.c | 116
+++++++++++++++++++++++++++++++++++++++
1 file changed, 116 insertions(+)
diff --git a/drivers/net/null/rte_eth_null.c
b/drivers/net/null/rte_eth_null.c index bf81b1b..b01f647 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -37,6 +37,8 @@
#include <rte_memcpy.h>
#include <rte_dev.h>
#include <rte_kvargs.h>
+#include <rte_eth_null.h>
+#include <rte_spinlock.h>
Hi Tomasz,
We don't have "rte_eth_null.h" at this point.
(The header file will be added next patch) Probably, we also need
"rte_pmd_null_version.map" to compile correctly.
(To make sure, please compile DPDK with "CONFIG_RTE_BUILD_SHARED_LIB=y"
option.)
Also, it seems 'rte_eth_null.h' should be included like below.
#include "rte_eth_null.h"
Without it, we cannot compile.
Thanks,
Tetsuya
Hi Tetsuya,
Hi Tomasz,

But "rte_eth_null.h" is used in " [PATCH v5 4/9] null: virtual dynamic
rss configuration".
It means we cannot compile DPDK before applying "[dpdk-dev,v5,5/9]".
When we need to use git-bisect, this will be problem.
So could you please keep DPDK being able to compile at any point?

Thanks,
Tetsuya,
Tomasz Kulasek
2015-09-30 14:05:01 UTC
Permalink
Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/null/Makefile | 2 +-
drivers/net/null/rte_eth_null.c | 2 +-
drivers/net/null/rte_eth_null.h | 40 +++++++++++++++++++++++++++++
drivers/net/null/rte_pmd_null_version.map | 7 +++++
4 files changed, 49 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/null/rte_eth_null.h

diff --git a/drivers/net/null/Makefile b/drivers/net/null/Makefile
index 96ba01c..2202389 100644
--- a/drivers/net/null/Makefile
+++ b/drivers/net/null/Makefile
@@ -51,7 +51,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += rte_eth_null.c
#
# Export include files
#
-SYMLINK-y-include +=
+SYMLINK-y-include += rte_eth_null.h

# this lib depends upon:
DEPDIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += lib/librte_mbuf
diff --git a/drivers/net/null/rte_eth_null.c b/drivers/net/null/rte_eth_null.c
index b01f647..56e4278 100644
--- a/drivers/net/null/rte_eth_null.c
+++ b/drivers/net/null/rte_eth_null.c
@@ -491,7 +491,7 @@ static const struct eth_dev_ops ops = {
.rss_hash_conf_get = eth_rss_hash_conf_get
};

-static int
+int
eth_dev_null_create(const char *name,
const unsigned numa_node,
unsigned packet_size,
diff --git a/drivers/net/null/rte_eth_null.h b/drivers/net/null/rte_eth_null.h
new file mode 100644
index 0000000..abada8c
--- /dev/null
+++ b/drivers/net/null/rte_eth_null.h
@@ -0,0 +1,40 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#ifndef RTE_ETH_NULL_H_
+#define RTE_ETH_NULL_H_
+
+int eth_dev_null_create(const char *name, const unsigned numa_node,
+ unsigned packet_size, unsigned packet_copy);
+
+#endif /* RTE_ETH_NULL_H_ */
diff --git a/drivers/net/null/rte_pmd_null_version.map b/drivers/net/null/rte_pmd_null_version.map
index ef35398..84b1d0f 100644
--- a/drivers/net/null/rte_pmd_null_version.map
+++ b/drivers/net/null/rte_pmd_null_version.map
@@ -2,3 +2,10 @@ DPDK_2.0 {

local: *;
};
+
+DPDK_2.2 {
+ global:
+
+ eth_dev_null_create;
+
+} DPDK_2.0;
--
1.7.9.5
Tetsuya Mukawa
2015-10-14 01:35:05 UTC
Permalink
Post by Tomasz Kulasek
---
drivers/net/null/Makefile | 2 +-
drivers/net/null/rte_eth_null.c | 2 +-
drivers/net/null/rte_eth_null.h | 40 +++++++++++++++++++++++++++++
drivers/net/null/rte_pmd_null_version.map | 7 +++++
4 files changed, 49 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/null/rte_eth_null.h
Acked-by: Tetsuya Mukawa <***@igel.co.jp>
Jastrzebski, MichalX K
2015-10-14 12:42:32 UTC
Permalink
-----Original Message-----
Sent: Wednesday, October 14, 2015 3:35 AM
Subject: Re: [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create
Post by Tomasz Kulasek
---
drivers/net/null/Makefile | 2 +-
drivers/net/null/rte_eth_null.c | 2 +-
drivers/net/null/rte_eth_null.h | 40
+++++++++++++++++++++++++++++
Post by Tomasz Kulasek
drivers/net/null/rte_pmd_null_version.map | 7 +++++
4 files changed, 49 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/null/rte_eth_null.h
Hi Tetsuya,
Thank You for a review and acking patches.
There are 5 remaining patches in a patch-set:

[dpdk-dev] [PATCH v5 1/9] bonding: rss dynamic configuration
[dpdk-dev] [PATCH v5 6/9] test: dynamic rss configuration
[dpdk-dev] [PATCH v5 7/9] bonding: per queue stats
[dpdk-dev] [PATCH v5 8/9] doc: fixed spellings and typos
[dpdk-dev] [PATCH v5 9/9] doc: dynamic rss configuration for bonding

Could You also review them and ack if You agree with that implementation, please?
Tetsuya Mukawa
2015-10-15 08:16:36 UTC
Permalink
Post by Kulasek, TomaszX
-----Original Message-----
Sent: Wednesday, October 14, 2015 3:35 AM
Subject: Re: [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create
Post by Tomasz Kulasek
---
drivers/net/null/Makefile | 2 +-
drivers/net/null/rte_eth_null.c | 2 +-
drivers/net/null/rte_eth_null.h | 40
+++++++++++++++++++++++++++++
Post by Tomasz Kulasek
drivers/net/null/rte_pmd_null_version.map | 7 +++++
4 files changed, 49 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/null/rte_eth_null.h
Hi Tetsuya,
Thank You for a review and acking patches.
[dpdk-dev] [PATCH v5 1/9] bonding: rss dynamic configuration
[dpdk-dev] [PATCH v5 6/9] test: dynamic rss configuration
[dpdk-dev] [PATCH v5 7/9] bonding: per queue stats
[dpdk-dev] [PATCH v5 8/9] doc: fixed spellings and typos
[dpdk-dev] [PATCH v5 9/9] doc: dynamic rss configuration for bonding
Could You also review them and ack if You agree with that implementation, please?
Hi MichalX,

I've checked above patches. But I am sorry that I am not a correct
person to ack your patches.
To review it correctly, at least knowledge about RSS and bonding PMD are
needed, but honestly I am not familiar with RSS.
Is it possible to find a correct person who can ack it?

BTW, I've checked the patch from the following point of view. It may
help other reviewers.
(I've checked after fixing null PMD issues I've found today, please
check an email I've sent to Tomasz)
- No fatal errors are reported by checkpatch.pl.
- Some style warnings are reported, but I guess we can ignore it.
- No compile error with gcc-4.6/4.7/4.8/4.9/5, clang and icc.
- No missing symbol.
- No compile error for docs.
- No compile error for examples.

Thanks,
Tetsuya
Jastrzebski, MichalX K
2015-10-15 10:35:28 UTC
Permalink
-----Original Message-----
Sent: Thursday, October 15, 2015 10:17 AM
To: Jastrzebski, MichalX K
Subject: Re: [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create
Post by Kulasek, TomaszX
-----Original Message-----
Sent: Wednesday, October 14, 2015 3:35 AM
Subject: Re: [dpdk-dev] [PATCH v5 5/9] null: export eth_dev_null_create
Post by Tomasz Kulasek
---
drivers/net/null/Makefile | 2 +-
drivers/net/null/rte_eth_null.c | 2 +-
drivers/net/null/rte_eth_null.h | 40
+++++++++++++++++++++++++++++
Post by Tomasz Kulasek
drivers/net/null/rte_pmd_null_version.map | 7 +++++
4 files changed, 49 insertions(+), 2 deletions(-)
create mode 100644 drivers/net/null/rte_eth_null.h
Hi Tetsuya,
Thank You for a review and acking patches.
[dpdk-dev] [PATCH v5 1/9] bonding: rss dynamic configuration
[dpdk-dev] [PATCH v5 6/9] test: dynamic rss configuration
[dpdk-dev] [PATCH v5 7/9] bonding: per queue stats
[dpdk-dev] [PATCH v5 8/9] doc: fixed spellings and typos
[dpdk-dev] [PATCH v5 9/9] doc: dynamic rss configuration for bonding
Could You also review them and ack if You agree with that implementation,
please?
Hi MichalX,
I've checked above patches. But I am sorry that I am not a correct
person to ack your patches.
To review it correctly, at least knowledge about RSS and bonding PMD are
needed, but honestly I am not familiar with RSS.
Is it possible to find a correct person who can ack it?
BTW, I've checked the patch from the following point of view. It may
help other reviewers.
(I've checked after fixing null PMD issues I've found today, please
check an email I've sent to Tomasz)
- No fatal errors are reported by checkpatch.pl.
- Some style warnings are reported, but I guess we can ignore it.
- No compile error with gcc-4.6/4.7/4.8/4.9/5, clang and icc.
- No missing symbol.
- No compile error for docs.
- No compile error for examples.
Thanks,
Tetsuya
Hi Tetsuya,
Thank You very much for verifications and Your comments.
I will ask maintainers in term of reviewing remaining patches.

Regarding an issue with rte_eth_null.h I think that reordering of patches
4 and 5 may be sufficient.

Best regards
Michal
Tomasz Kulasek
2015-09-30 14:05:02 UTC
Permalink
Signed-off-by: Tomasz Kulasek <***@intel.com>
---
app/test/Makefile | 8 +
app/test/test_link_bonding_rssconf.c | 679 ++++++++++++++++++++++++++++++++++
2 files changed, 687 insertions(+)
create mode 100644 app/test/test_link_bonding_rssconf.c

diff --git a/app/test/Makefile b/app/test/Makefile
index 294618f..c122f28 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -138,6 +138,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_ACL) += test_acl.c
ifeq ($(CONFIG_RTE_LIBRTE_PMD_RING),y)
SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding.c
SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_mode4.c
+SRCS-$(CONFIG_RTE_LIBRTE_PMD_BOND) += test_link_bonding_rssconf.c
endif

SRCS-$(CONFIG_RTE_LIBRTE_PMD_RING) += test_pmd_ring.c
@@ -168,6 +169,13 @@ ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
LDLIBS += -lrte_pmd_ring
endif
endif
+ifneq ($(CONFIG_RTE_LIBRTE_PMD_NULL),y)
+$(error Link bonding rssconf tests require CONFIG_RTE_LIBRTE_PMD_NULL=y)
+else
+ifeq ($(CONFIG_RTE_BUILD_SHARED_LIB),y)
+LDLIBS += -lrte_pmd_null
+endif
+endif
endif

include $(RTE_SDK)/mk/rte.app.mk
diff --git a/app/test/test_link_bonding_rssconf.c b/app/test/test_link_bonding_rssconf.c
new file mode 100644
index 0000000..e6714b4
--- /dev/null
+++ b/app/test/test_link_bonding_rssconf.c
@@ -0,0 +1,679 @@
+/*-
+ * BSD LICENSE
+ *
+ * Copyright(c) 2015 Intel Corporation. All rights reserved.
+ * All rights reserved.
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * * Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * * Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * * Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ */
+
+#include <string.h>
+#include <stdarg.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <inttypes.h>
+#include <errno.h>
+#include <rte_cycles.h>
+#include <sys/queue.h>
+
+#include <rte_byteorder.h>
+#include <rte_common.h>
+#include <rte_debug.h>
+#include <rte_ethdev.h>
+#include <rte_log.h>
+#include <rte_lcore.h>
+#include <rte_memory.h>
+
+#include <rte_string_fns.h>
+#include <rte_errno.h>
+#include <rte_eth_bond.h>
+#include <rte_eth_null.h>
+
+#include "test.h"
+
+#define SLAVE_COUNT (4)
+
+#define RXTX_RING_SIZE 1024
+#define RXTX_QUEUE_COUNT 4
+
+#define BONDED_DEV_NAME ("rssconf_bond_dev")
+
+#define SLAVE_DEV_NAME_FMT ("rssconf_slave%d")
+#define SLAVE_RXTX_QUEUE_FMT ("rssconf_slave%d_q%d")
+
+#define NUM_MBUFS 8191
+#define MBUF_SIZE (1600 + sizeof(struct rte_mbuf) + RTE_PKTMBUF_HEADROOM)
+#define MBUF_CACHE_SIZE 250
+#define BURST_SIZE 32
+
+#define INVALID_SOCKET_ID (-1)
+#define INVALID_PORT_ID (0xFF)
+#define INVALID_BONDING_MODE (-1)
+
+struct slave_conf {
+ uint8_t port_id;
+ struct rte_eth_dev_info dev_info;
+
+ struct rte_eth_rss_conf rss_conf;
+ uint8_t rss_key[40];
+ struct rte_eth_rss_reta_entry64 reta_conf[512 / RTE_RETA_GROUP_SIZE];
+
+ uint8_t is_slave;
+ struct rte_ring *rxtx_queue[RXTX_QUEUE_COUNT];
+};
+
+struct link_bonding_rssconf_unittest_params {
+ uint8_t bond_port_id;
+ struct rte_eth_dev_info bond_dev_info;
+ struct rte_eth_rss_reta_entry64 bond_reta_conf[512 / RTE_RETA_GROUP_SIZE];
+ struct slave_conf slave_ports[SLAVE_COUNT];
+
+ struct rte_mempool *mbuf_pool;
+};
+
+static struct link_bonding_rssconf_unittest_params test_params = {
+ .bond_port_id = INVALID_PORT_ID,
+ .slave_ports = {
+ [0 ... SLAVE_COUNT - 1] = { .port_id = INVALID_PORT_ID, .is_slave = 0}
+ },
+ .mbuf_pool = NULL,
+};
+
+/**
+ * Default port configuration with RSS turned off
+ */
+static struct rte_eth_conf default_pmd_conf = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_NONE,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled */
+ .hw_ip_checksum = 0, /**< IP checksum offload enabled */
+ .hw_vlan_filter = 0, /**< VLAN filtering disabled */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled */
+ .hw_strip_crc = 0, /**< CRC stripped by hardware */
+ },
+ .txmode = {
+ .mq_mode = ETH_MQ_TX_NONE,
+ },
+ .lpbk_mode = 0,
+};
+
+static struct rte_eth_conf rss_pmd_conf = {
+ .rxmode = {
+ .mq_mode = ETH_MQ_RX_RSS,
+ .max_rx_pkt_len = ETHER_MAX_LEN,
+ .split_hdr_size = 0,
+ .header_split = 0, /**< Header Split disabled */
+ .hw_ip_checksum = 0, /**< IP checksum offload enabled */
+ .hw_vlan_filter = 0, /**< VLAN filtering disabled */
+ .jumbo_frame = 0, /**< Jumbo Frame Support disabled */
+ .hw_strip_crc = 0, /**< CRC stripped by hardware */
+ },
+ .txmode = {
+ .mq_mode = ETH_MQ_TX_NONE,
+ },
+ .rx_adv_conf = {
+ .rss_conf = {
+ .rss_key = NULL,
+ .rss_hf = ETH_RSS_IPV6,
+ },
+ },
+ .lpbk_mode = 0,
+};
+
+#define FOR_EACH(_i, _item, _array, _size) \
+ for (_i = 0, _item = &_array[0]; _i < _size && (_item = &_array[_i]); _i++)
+
+/* Macro for iterating over every port that can be used as a slave
+ * in this test.
+ * _i variable used as an index in test_params->slave_ports
+ * _slave pointer to &test_params->slave_ports[_idx]
+ */
+#define FOR_EACH_PORT(_i, _port) \
+ FOR_EACH(_i, _port, test_params.slave_ports, \
+ RTE_DIM(test_params.slave_ports))
+
+static int
+configure_ethdev(uint8_t port_id, struct rte_eth_conf *eth_conf, uint8_t start)
+{
+ int rxq, txq;
+
+ TEST_ASSERT(rte_eth_dev_configure(port_id, RXTX_QUEUE_COUNT,
+ RXTX_QUEUE_COUNT, eth_conf) == 0, "Failed to configure device %u",
+ port_id);
+
+ for (rxq = 0; rxq < RXTX_QUEUE_COUNT; rxq++) {
+ TEST_ASSERT(rte_eth_rx_queue_setup(port_id, rxq, RXTX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id), NULL,
+ test_params.mbuf_pool) == 0, "Failed to setup rx queue.");
+ }
+
+ for (txq = 0; txq < RXTX_QUEUE_COUNT; txq++) {
+ TEST_ASSERT(rte_eth_tx_queue_setup(port_id, txq, RXTX_RING_SIZE,
+ rte_eth_dev_socket_id(port_id), NULL) == 0,
+ "Failed to setup tx queue.");
+ }
+
+ if (start) {
+ TEST_ASSERT(rte_eth_dev_start(port_id) == 0,
+ "Failed to start device (%d).", port_id);
+ }
+
+ return 0;
+}
+
+/**
+ * Remove all slaves from bonding
+ */
+static int
+remove_slaves(void)
+{
+ unsigned n;
+ struct slave_conf *port;
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ if (port->is_slave) {
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(
+ test_params.bond_port_id, port->port_id),
+ "Cannot remove slave %d from bonding", port->port_id);
+ port->is_slave = 0;
+ }
+ }
+
+ return 0;
+}
+
+static int
+remove_slaves_and_stop_bonded_device(void)
+{
+ TEST_ASSERT_SUCCESS(remove_slaves(), "Removing slaves");
+ rte_eth_dev_stop(test_params.bond_port_id);
+ return TEST_SUCCESS;
+}
+
+/**
+ * Add all slaves to bonding
+ */
+static int
+bond_slaves(void)
+{
+ unsigned n;
+ struct slave_conf *port;
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ if (!port->is_slave) {
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id,
+ port->port_id), "Cannot attach slave %d to the bonding",
+ port->port_id);
+ port->is_slave = 1;
+ }
+ }
+
+ return 0;
+}
+
+/**
+ * Set all RETA values in port_id to value
+ */
+static int
+reta_set(uint8_t port_id, uint8_t value, int reta_size)
+{
+ struct rte_eth_rss_reta_entry64 reta_conf[512/RTE_RETA_GROUP_SIZE];
+ int i, j;
+
+ for (i = 0; i < reta_size / RTE_RETA_GROUP_SIZE; i++) {
+ /* select all fields to set */
+ reta_conf[i].mask = ~0LL;
+ for (j = 0; j < RTE_RETA_GROUP_SIZE; j++)
+ reta_conf[i].reta[j] = value;
+ }
+
+ return rte_eth_dev_rss_reta_update(port_id, reta_conf, reta_size);
+}
+
+/**
+ * Check if slaves RETA is synchronized with bonding port. Returns 1 if slave
+ * port is synced with bonding port.
+ */
+static int
+reta_check_synced(struct slave_conf *port)
+{
+ unsigned i;
+
+ for (i = 0; i < test_params.bond_dev_info.reta_size;
+ i++) {
+
+ int index = i / RTE_RETA_GROUP_SIZE;
+ int shift = i % RTE_RETA_GROUP_SIZE;
+
+ if (port->reta_conf[index].reta[shift] !=
+ test_params.bond_reta_conf[index].reta[shift])
+ return 0;
+
+ }
+
+ return 1;
+}
+
+/**
+ * Fetch bonding ports RETA
+ */
+static int
+bond_reta_fetch(void) {
+ unsigned j;
+
+ for (j = 0; j < test_params.bond_dev_info.reta_size / RTE_RETA_GROUP_SIZE;
+ j++)
+ test_params.bond_reta_conf[j].mask = ~0LL;
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(test_params.bond_port_id,
+ test_params.bond_reta_conf, test_params.bond_dev_info.reta_size),
+ "Cannot take bonding ports RSS configuration");
+ return 0;
+}
+
+/**
+ * Fetch slaves RETA
+ */
+static int
+slave_reta_fetch(struct slave_conf *port) {
+ unsigned j;
+
+ for (j = 0; j < port->dev_info.reta_size / RTE_RETA_GROUP_SIZE; j++)
+ port->reta_conf[j].mask = ~0LL;
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_rss_reta_query(port->port_id,
+ port->reta_conf, port->dev_info.reta_size),
+ "Cannot take bonding ports RSS configuration");
+ return 0;
+}
+
+/**
+ * Remove and add slave to check if slaves configuration is synced with
+ * the bonding ports values after adding new slave.
+ */
+static int
+slave_remove_and_add(void)
+{
+ struct slave_conf *port = &(test_params.slave_ports[0]);
+
+ /* 1. Remove first slave from bonding */
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_remove(test_params.bond_port_id,
+ port->port_id), "Cannot remove slave #d from bonding");
+
+ /* 2. Change removed (ex-)slave and bonding configuration to different
+ * values
+ */
+ reta_set(test_params.bond_port_id, 1, test_params.bond_dev_info.reta_size);
+ bond_reta_fetch();
+
+ reta_set(port->port_id, 2, port->dev_info.reta_size);
+ slave_reta_fetch(port);
+
+ TEST_ASSERT(reta_check_synced(port) == 0,
+ "Removed slave didn't should be synchronized with bonding port");
+
+ /* 3. Add (ex-)slave and check if configuration changed*/
+ TEST_ASSERT_SUCCESS(rte_eth_bond_slave_add(test_params.bond_port_id,
+ port->port_id), "Cannot add slave");
+
+ bond_reta_fetch();
+ slave_reta_fetch(port);
+
+ return reta_check_synced(port);
+}
+
+/**
+ * Test configuration propagation over slaves.
+ */
+static int
+test_propagate(void)
+{
+ unsigned i;
+ uint8_t n;
+ struct slave_conf *port;
+ uint8_t bond_rss_key[40];
+ struct rte_eth_rss_conf bond_rss_conf;
+
+ int retval = 0;
+ uint64_t rss_hf = 0;
+ uint64_t default_rss_hf = 0;
+
+ rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info);
+
+ /*
+ * Test hash function propagation
+ */
+ for (i = 0; i < sizeof(test_params.bond_dev_info.flow_type_rss_offloads)*8;
+ i++) {
+
+ rss_hf = test_params.bond_dev_info.flow_type_rss_offloads & (1<<i);
+ if (rss_hf) {
+ bond_rss_conf.rss_key = NULL;
+ bond_rss_conf.rss_hf = rss_hf;
+
+ retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id,
+ &bond_rss_conf);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set slaves hash function");
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ retval = rte_eth_dev_rss_hash_conf_get(port->port_id,
+ &port->rss_conf);
+ TEST_ASSERT_SUCCESS(retval,
+ "Cannot take slaves RSS configuration");
+
+ TEST_ASSERT(port->rss_conf.rss_hf == rss_hf,
+ "Hash function not propagated for slave %d",
+ port->port_id);
+ }
+
+ default_rss_hf = rss_hf;
+ }
+
+ }
+
+ /*
+ * Test key propagation
+ */
+ for (i = 1; i < 10; i++) {
+
+ /* Set all keys to zero */
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ memset(port->rss_conf.rss_key, 0, 40);
+ retval = rte_eth_dev_rss_hash_update(port->port_id,
+ &port->rss_conf);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RSS keys");
+ }
+
+ memset(bond_rss_key, i, sizeof(bond_rss_key));
+ bond_rss_conf.rss_hf = default_rss_hf,
+ bond_rss_conf.rss_key = bond_rss_key;
+ bond_rss_conf.rss_key_len = 40;
+
+ retval = rte_eth_dev_rss_hash_update(test_params.bond_port_id,
+ &bond_rss_conf);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set bonded port RSS keys");
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ retval = rte_eth_dev_rss_hash_conf_get(port->port_id,
+ &(port->rss_conf));
+
+ TEST_ASSERT_SUCCESS(retval,
+ "Cannot take slaves RSS configuration");
+
+ /* compare keys */
+ retval = memcmp(port->rss_conf.rss_key, bond_rss_key,
+ sizeof(bond_rss_key));
+ TEST_ASSERT(retval == 0, "Key value not propagated for slave %d",
+ port->port_id);
+ }
+ }
+
+ /*
+ * Test RETA propagation
+ */
+ for (i = 0; i < RXTX_QUEUE_COUNT; i++) {
+
+ /* Set all keys to zero */
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+ retval = reta_set(port->port_id, (i + 1) % RXTX_QUEUE_COUNT,
+ port->dev_info.reta_size);
+ TEST_ASSERT_SUCCESS(retval, "Cannot set slaves RETA");
+ }
+
+ TEST_ASSERT_SUCCESS(reta_set(test_params.bond_port_id,
+ i % RXTX_QUEUE_COUNT, test_params.bond_dev_info.reta_size),
+ "Cannot set bonded port RETA");
+
+ bond_reta_fetch();
+
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ slave_reta_fetch(port);
+ TEST_ASSERT(reta_check_synced(port) == 1, "RETAs inconsistent");
+ }
+ }
+
+ return TEST_SUCCESS;
+}
+
+/**
+ * Test propagation logic, when RX_RSS mq_mode is turned on for bonding port
+ */
+static int
+test_rss(void)
+{
+ /**
+ * Configure bonding port in RSS mq mode
+ */
+ TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id,
+ &rss_pmd_conf, 0), "Failed to configure bonding device\n");
+
+ rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info);
+
+ TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed");
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id),
+ "Failed to start bonding port (%d).", test_params.bond_port_id);
+
+ TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed");
+
+ TEST_ASSERT(slave_remove_and_add() == 1, "New slave should be synced");
+
+ remove_slaves_and_stop_bonded_device();
+
+ return TEST_SUCCESS;
+}
+
+/**
+ * Test propagation logic, when RX_RSS mq_mode is turned off for bonding port
+ */
+static int
+test_rss_lazy(void)
+{
+ TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id,
+ &default_pmd_conf, 0), "Failed to configure bonding device\n");
+
+ rte_eth_dev_info_get(test_params.bond_port_id, &test_params.bond_dev_info);
+
+ TEST_ASSERT_SUCCESS(bond_slaves(), "Bonding slaves failed");
+
+ TEST_ASSERT_SUCCESS(rte_eth_dev_start(test_params.bond_port_id),
+ "Failed to start bonding port (%d).", test_params.bond_port_id);
+
+ TEST_ASSERT_SUCCESS(test_propagate(), "Propagation test failed");
+
+ TEST_ASSERT(slave_remove_and_add() == 0, "New slave shouldn't be synced");
+
+ remove_slaves_and_stop_bonded_device();
+
+ return TEST_SUCCESS;
+}
+
+static int
+test_setup(void)
+{
+ unsigned n;
+ int retval;
+ int port_id;
+ char name[256];
+ struct slave_conf *port;
+
+ if (test_params.mbuf_pool == NULL) {
+
+ test_params.mbuf_pool = rte_mempool_create("RSS_MBUF_POOL", NUM_MBUFS *
+ SLAVE_COUNT, MBUF_SIZE, MBUF_CACHE_SIZE,
+ sizeof(struct rte_pktmbuf_pool_private), rte_pktmbuf_pool_init,
+ NULL, rte_pktmbuf_init, NULL, rte_socket_id(), 0);
+
+ TEST_ASSERT(test_params.mbuf_pool != NULL,
+ "rte_mempool_create failed\n");
+ }
+
+ /* Create / initialize ring eth devs. */
+ FOR_EACH_PORT(n, port) {
+ port = &test_params.slave_ports[n];
+
+ port_id = rte_eth_dev_count();
+ snprintf(name, sizeof(name), SLAVE_DEV_NAME_FMT, port_id);
+
+ retval = eth_dev_null_create(name, 0, 64, 0);
+ TEST_ASSERT_SUCCESS(retval, "Failed to create null device '%s'\n",
+ name);
+
+ port->port_id = port_id;
+
+ port->rss_conf.rss_key = port->rss_key;
+ port->rss_conf.rss_key_len = 40;
+
+ retval = configure_ethdev(port->port_id, &default_pmd_conf, 0);
+ TEST_ASSERT_SUCCESS(retval, "Failed to configure virtual ethdev %s\n",
+ name);
+
+ rte_eth_dev_info_get(port->port_id, &port->dev_info);
+ }
+
+ if (test_params.bond_port_id == INVALID_PORT_ID) {
+ retval = rte_eth_bond_create(BONDED_DEV_NAME, 0, rte_socket_id());
+
+ TEST_ASSERT(retval >= 0, "Failed to create bonded ethdev %s",
+ BONDED_DEV_NAME);
+
+ test_params.bond_port_id = retval;
+
+ TEST_ASSERT_SUCCESS(configure_ethdev(test_params.bond_port_id,
+ &default_pmd_conf, 0), "Failed to configure bonding device\n");
+
+ rte_eth_dev_info_get(test_params.bond_port_id,
+ &test_params.bond_dev_info);
+ }
+
+ return TEST_SUCCESS;
+}
+
+static int
+testsuite_teardown(void)
+{
+ struct slave_conf *port;
+ uint8_t i;
+
+ /* Only stop ports.
+ * Any cleanup/reset state is done when particular test is
+ * started. */
+
+ rte_eth_dev_stop(test_params.bond_port_id);
+
+ FOR_EACH_PORT(i, port)
+ rte_eth_dev_stop(port->port_id);
+
+ return 0;
+}
+
+static int
+check_environment(void)
+{
+ return TEST_SUCCESS;
+}
+
+static int
+test_rssconf_executor(int (*test_func)(void))
+{
+ int test_result;
+
+ /* Check if environment is clean. Fail to launch a test if there was
+ * a critical error before that prevented to reset environment. */
+ TEST_ASSERT_SUCCESS(check_environment(),
+ "Refusing to launch test in dirty environment.");
+
+ RTE_VERIFY(test_func != NULL);
+ test_result = (*test_func)();
+
+ /* If test succeed check if environment wast left in good condition. */
+ if (test_result == TEST_SUCCESS)
+ test_result = check_environment();
+
+ /* Reset environment in case test failed to do that. */
+ if (test_result != TEST_SUCCESS) {
+ TEST_ASSERT_SUCCESS(remove_slaves_and_stop_bonded_device(),
+ "Failed to stop bonded device");
+ }
+
+ return test_result;
+}
+
+static int
+test_setup_wrapper(void)
+{
+ return test_rssconf_executor(&test_setup);
+}
+
+static int
+test_rss_wrapper(void)
+{
+ return test_rssconf_executor(&test_rss);
+}
+
+static int
+test_rss_lazy_wrapper(void)
+{
+ return test_rssconf_executor(&test_rss_lazy);
+}
+
+static struct unit_test_suite link_bonding_rssconf_test_suite = {
+ .suite_name = "RSS Dynamic Configuration for Bonding Unit Test Suite",
+ .teardown = testsuite_teardown,
+ .unit_test_cases = {
+ TEST_CASE_NAMED("test_setup", test_setup_wrapper),
+ TEST_CASE_NAMED("test_rss", test_rss_wrapper),
+ TEST_CASE_NAMED("test_rss_lazy", test_rss_lazy_wrapper),
+ { NULL, NULL, NULL, NULL, NULL } /**< NULL terminate unit test array */
+ }
+};
+
+static int
+test_link_bonding_rssconf(void)
+{
+ return unit_test_suite_runner(&link_bonding_rssconf_test_suite);
+}
+
+static struct test_command test_link_bonding_rssconf_cmd = {
+ .command = "link_bonding_rssconf_autotest",
+ .callback = test_link_bonding_rssconf,
+};
+
+REGISTER_TEST_COMMAND(test_link_bonding_rssconf_cmd);
--
1.7.9.5
Tomasz Kulasek
2015-09-30 14:05:03 UTC
Permalink
This patch fills bonding port's stats with a sum of corresponding values
taken from bonded slaves, when stats are requested for bonding port.

v5 changes:
- removed queue_stats_mapping_set from eth_dev_ops of bonding device

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
drivers/net/bonding/rte_eth_bond_pmd.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/drivers/net/bonding/rte_eth_bond_pmd.c b/drivers/net/bonding/rte_eth_bond_pmd.c
index 2880f5c..eecb381 100644
--- a/drivers/net/bonding/rte_eth_bond_pmd.c
+++ b/drivers/net/bonding/rte_eth_bond_pmd.c
@@ -1801,7 +1801,7 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
{
struct bond_dev_private *internals = dev->data->dev_private;
struct rte_eth_stats slave_stats;
- int i;
+ int i, j;

for (i = 0; i < internals->slave_count; i++) {
rte_eth_stats_get(internals->slaves[i].port_id, &slave_stats);
@@ -1820,6 +1820,15 @@ bond_ethdev_stats_get(struct rte_eth_dev *dev, struct rte_eth_stats *stats)
stats->rx_pause_xon += slave_stats.rx_pause_xon;
stats->tx_pause_xoff += slave_stats.tx_pause_xoff;
stats->rx_pause_xoff += slave_stats.rx_pause_xoff;
+
+ for (j = 0; j < RTE_ETHDEV_QUEUE_STAT_CNTRS; j++) {
+ stats->q_ipackets[j] += slave_stats.q_ipackets[j];
+ stats->q_opackets[j] += slave_stats.q_opackets[j];
+ stats->q_ibytes[j] += slave_stats.q_ibytes[j];
+ stats->q_obytes[j] += slave_stats.q_obytes[j];
+ stats->q_errors[j] += slave_stats.q_errors[j];
+ }
+
}
}
--
1.7.9.5
Tomasz Kulasek
2015-09-30 14:05:04 UTC
Permalink
Signed-off-by: Tomasz Kulasek <***@intel.com>
---
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
index 96e554f..03baf90 100644
--- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
+++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
@@ -188,7 +188,7 @@ conditions are not met. If a user wishes to monitor individual slaves then they
must register callbacks with that slave directly.

The link bonding library also supports devices which do not implement link
-status change interrupts, this is achieve by polling the devices link status at
+status change interrupts, this is achieved by polling the devices link status at
a defined period which is set using the ``rte_eth_bond_link_monitoring_set``
API, the default polling interval is 10ms. When a device is added as a slave to
a bonding device it is determined using the ``RTE_PCI_DRV_INTR_LSC`` flag
@@ -286,7 +286,7 @@ and UDP protocols for load balancing.
Using Link Bonding Devices
--------------------------

-The librte_pmd_bond library support two modes of device creation, the libraries
+The librte_pmd_bond library supports two modes of device creation, the libraries
export full C API or using the EAL command line to statically configure link
bonding devices at application startup. Using the EAL option it is possible to
use link bonding functionality transparently without specific knowledge of the
@@ -299,7 +299,7 @@ Using the Poll Mode Driver from an Application

Using the librte_pmd_bond libraries API it is possible to dynamically create
and manage link bonding device from within any application. Link bonding
-device are created using the ``rte_eth_bond_create`` API which requires a
+devices are created using the ``rte_eth_bond_create`` API which requires a
unique device name, the link bonding mode to initial the device in and finally
the socket Id which to allocate the devices resources onto. After successful
creation of a bonding device it must be configured using the generic Ethernet
@@ -362,7 +362,7 @@ The different options are:
mode=2

* slave: Defines the PMD device which will be added as slave to the bonded
- device. This option can be selected multiple time, for each device to be
+ device. This option can be selected multiple times, for each device to be
added as a slave. Physical devices should be specified using their PCI
address, in the format domain:bus:devid.function
--
1.7.9.5
Tomasz Kulasek
2015-09-30 14:05:05 UTC
Permalink
Documentation update about implementation details and requirements for
Dynamic RSS Configuration for Bonding.

Signed-off-by: Tomasz Kulasek <***@intel.com>
---
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 34 ++++++++++++++++++--
1 file changed, 32 insertions(+), 2 deletions(-)

diff --git a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
index 03baf90..46f0296 100644
--- a/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
+++ b/doc/guides/prog_guide/link_bonding_poll_mode_drv_lib.rst
@@ -1,5 +1,5 @@
.. BSD LICENSE
- Copyright(c) 2010-2014 Intel Corporation. All rights reserved.
+ Copyright(c) 2010-2015 Intel Corporation. All rights reserved.
All rights reserved.

Redistribution and use in source and binary forms, with or without
@@ -173,7 +173,28 @@ After a slave device is added to a bonded device slave is stopped using
``rte_eth_dev_stop`` and then reconfigured using ``rte_eth_dev_configure``
the RX and TX queues are also reconfigured using ``rte_eth_tx_queue_setup`` /
``rte_eth_rx_queue_setup`` with the parameters use to configure the bonding
-device.
+device. If RSS is enabled for bonding device, this mode is also enabled on new
+slave and configured as well.
+
+Setting up multi-queue mode for bonding device to RSS, makes it fully
+RSS-capable, so all slaves are synchronized with its configuration. This mode is
+intended to provide RSS configuration on slaves transparent for client
+application implementation.
+
+Bonding device stores its own version of RSS settings i.e. RETA, RSS hash
+function and RSS key, used to set up its slaves. That let to define the meaning
+of RSS configuration of bonding device as desired configuration of whole bonding
+(as one unit), without pointing any of slave inside. It is required to ensure
+consistency and made it more errorproof.
+
+RSS hash function set for bonding device, is a maximal set of RSS hash functions
+supported by all bonded slaves. RETA size is a GCD of all its RETA's sizes, so
+it can be easily used as a pattern providing expected behavior, even if slave
+RETAs' sizes are different. If RSS Key is not set for bonded device, it's not
+changed on the slaves and default key for device is used.
+
+All settings are managed through the bonding port API and always are propagated
+in one direction (from bonding to slaves).

Link Status Change Interrupts / Polling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -207,6 +228,15 @@ these parameters.
A bonding device must have a minimum of one slave before the bonding device
itself can be started.

+To use a bonding device dynamic RSS configuration feature effectively, it is
+also required, that all slaves should be RSS-capable and support, at least one
+common hash function available for each of them. Changing RSS key is only
+possible, when all slave devices support the same key size.
+
+To prevent inconsistency on how slaves process packets, once a device is added
+to a bonding device, RSS configuration should be managed through the bonding
+device API, and not directly on the slave.
+
Like all other PMD, all functions exported by a PMD are lock-free functions
that are assumed not to be invoked in parallel on different logical cores to
work on the same target object.
--
1.7.9.5
Tomasz Kulasek
2015-10-16 10:00:42 UTC
Permalink
OVERVIEW
--------
1) Setting .rxmode.mq_mode for bonding device to ETH_MQ_RX_RSS makes bonding
device fully RSS-capable, so all slaves are synchronized with its configuration.
This mode is intended to provide RSS configuration as known from "dynamic RSS
configuration for one port" and made slaves transparent for client application
implementation.

2) If .rxmode.mq_mode for bonding device isn't ETH_MQ_RX_RSS, slaves are not
synchronized. That provides an ability to configure them manually. This mode may
be useful when application wants to manage RSS in an unusual way and the
consistency of RSS configuration for slaves isn't required.

Turning on/off RSS mode for slaves when bonding is started is not possible.
Other RSS configuration is propagated over slaves, when bonding device API is
used to do it.

v6 changes:
- patchset reordered
- fixed forward dependency between patch 4/9 and 5/9

v5 changes:
- updated to DPDK 2.2
- removed copyright change from null device source
- removed queue_stats_mapping_set from eth_dev_ops of bonding device
- null pmd cleanups (removed unnecessary malloc, replaced memcpy with
rte_memcpy)
- fixed queues number configuration in null pmd

v4 changes:
- fixed copy-paste error,
- removed example application as too complex and introducing a new
dependency,
- addapted null pmd to be used as testing device for dynamic RSS configuration,
- addapted test units to use null pmd instead of ring pmd,
- ring pmd is not used and changed in this patchset

v3 changes:
- checkpatch cleanups

v2 changes:
- added support for keys other than 40 bytes long,
- now, if RSS key is not set for bonding, it is not set also for slaves,
- fix - full initial RSS configuration before any slave is added was not
possible due to the initially zeroed flow_type_rss_offloads for bonding,
- fix - changed error to warning when slave is synchronizing due to the
bonding's initial configuration (to allow use slaves' drivers not supporting
dynamic RSS configuration in bonding),
- some code cleanups,
- updated documentation,

Tomasz Kulasek (9):
bonding: rss dynamic configuration
null: fix segfault when null_pmd added to bonding
null: extend number of virtual queues
null: export eth_dev_null_create
null: virtual dynamic rss configuration
test: dynamic rss configuration
bonding: per queue stats
doc: fixed spellings and typos
doc: dynamic rss configuration for bonding

app/test/Makefile | 8 +
app/test/test_link_bonding_rssconf.c | 679 ++++++++++++++++++++
.../prog_guide/link_bonding_poll_mode_drv_lib.rst | 42 +-
drivers/net/bonding/rte_eth_bond_api.c | 28 +
drivers/net/bonding/rte_eth_bond_pmd.c | 216 ++++++-
drivers/net/bonding/rte_eth_bond_private.h | 12 +
drivers/net/null/Makefile | 2 +-
drivers/net/null/rte_eth_null.c | 149 ++++-
drivers/net/null/rte_eth_null.h | 40 ++
drivers/net/null/rte_pmd_null_version.map | 7 +
10 files changed, 1151 insertions(+), 32 deletions(-)
create mode 100644 app/test/test_link_bonding_rssconf.c
create mode 100644 drivers/net/null/rte_eth_null.h
--
1.7.9.5
Loading...