Eric Dumazet 93f154b594
net: release dst entry in dev_hard_start_xmit() ...
One point of contention in high network loads is the dst_release() performed
when a transmited skb is freed. This is because NIC tx completion calls
dev_kree_skb() long after original call to dev_queue_xmit(skb).
CPU cache is cold and the atomic op in dst_release() stalls. On SMP, this is
quite visible if one CPU is 100% handling softirqs for a network device,
since dst_clone() is done by other cpus, involving cache line ping pongs.
It seems right place to release dst is in dev_hard_start_xmit(), for most
devices but ones that are virtual, and some exceptions.
David Miller suggested to define a new device flag, set in alloc_netdev_mq()
(so that most devices set it at init time), and carefuly unset in devices
which dont want a NULL skb->dst in their ndo_start_xmit().
List of devices that must clear this flag is :
- loopback device, because it calls netif_rx() and quoting Patrick :
"ip_route_input() doesn't accept loopback addresses, so loopback packets
already need to have a dst_entry attached."
- appletalk/ipddp.c : needs skb->dst in its xmit function
- And all devices that call again dev_queue_xmit() from their xmit function
(as some classifiers need skb->dst) : bonding, vlan, macvlan, eql, ifb, hdlc_fr
Signed-off-by: Eric Dumazet <dada1@cosmosbay.com >
Signed-off-by: David S. Miller <davem@davemloft.net >
2009-05-18 22:19:19 -07:00
2009-01-06 18:10:26 -08:00
2009-05-18 15:41:42 -07:00
2009-01-08 15:14:17 -08:00
2009-03-26 18:14:15 +01:00
2009-04-07 10:23:34 +01:00
2009-03-18 17:30:51 -04:00
2009-04-08 17:45:02 -07:00
2009-03-18 19:45:11 -07:00
2009-04-22 16:54:41 -04:00
2009-04-06 07:01:56 -07:00
2009-05-05 12:00:53 -07:00
2008-11-04 14:27:15 +01:00
2009-03-26 18:14:14 +01:00
2009-03-26 16:11:41 -07:00
2009-04-06 09:48:20 -07:00
2009-03-31 15:09:39 +11:00
2009-04-28 18:58:06 +01:00
2009-05-18 21:08:20 -07:00
2009-03-05 14:39:32 -05:00
2009-05-02 16:35:08 -07:00
2009-01-30 23:34:34 +05:30
2009-01-30 23:36:36 +05:30
2009-05-18 21:08:20 -07:00
2008-12-22 18:30:29 +00:00
2009-03-24 16:38:23 -07:00
2009-01-02 10:19:37 -08:00
2009-04-21 00:35:47 -04:00
2009-03-26 18:14:14 +01:00
2009-01-30 23:38:03 +05:30
2008-12-29 08:29:50 +01:00
2009-01-21 14:02:19 -08:00
2009-03-25 09:13:25 -07:00
2009-02-08 09:56:11 -08:00
2009-05-15 14:14:56 -04:00
2009-01-30 23:38:37 +05:30
2009-01-29 16:00:33 -05:00
2009-01-15 16:39:41 -08:00
2008-12-03 22:12:38 -08:00
2009-01-30 23:39:08 +05:30
2008-12-03 22:12:38 -08:00
2009-01-09 11:52:14 -08:00
2009-03-26 18:14:15 +01:00
2009-03-26 18:14:15 +01:00
2009-02-12 09:40:14 +11:00
2009-04-01 08:59:23 -07:00
2009-01-30 23:39:42 +05:30
2009-04-01 08:59:23 -07:00
2009-01-08 08:31:12 -08:00
2009-04-06 08:04:53 -07:00
2009-01-08 15:37:43 +00:00
2009-01-30 23:40:06 +05:30
2009-05-02 15:36:10 -07:00
2009-04-22 08:35:10 +02:00
2008-12-30 09:05:13 +10:30
2009-04-23 10:06:35 +01:00
2009-05-02 16:48:32 -07:00
2009-02-19 09:00:35 +01:00
2009-01-04 13:33:20 -08:00
2009-04-01 08:59:14 -07:00
2008-11-28 12:38:38 +01:00
2009-03-28 13:30:43 -07:00
2009-04-16 07:47:49 -07:00
2008-11-12 17:17:18 -08:00
2009-04-13 08:32:28 -07:00
2009-01-15 16:39:41 -08:00
2009-01-30 23:44:08 +05:30
2008-11-07 22:56:00 -08:00
2009-04-03 15:24:35 -07:00
2009-01-30 23:44:41 +05:30
2009-03-27 14:51:13 +01:00
2009-01-30 22:35:29 +01:00
2009-04-21 13:41:47 -07:00
2009-03-26 18:14:15 +01:00
2009-03-26 18:14:14 +01:00
2008-11-09 11:17:33 -08:00
2009-02-03 19:32:24 +05:30
2009-01-23 22:28:48 -08:00
2009-04-04 14:20:34 -07:00
2009-01-02 12:19:34 -08:00
2009-03-26 18:14:23 +01:00
2009-03-12 21:33:03 +00:00
2009-04-07 09:05:35 -07:00
2009-04-03 14:53:32 -07:00
2009-03-24 16:38:26 -07:00
2009-04-02 19:05:00 -07:00
2009-03-09 15:07:33 -04:00
2009-01-01 10:12:30 +10:30
2009-04-02 19:04:57 -07:00
2008-12-25 11:01:43 +11:00
2008-11-24 18:57:41 -05:00
2009-03-04 15:05:33 +08:00
2009-03-26 18:14:14 +01:00
2009-03-21 13:31:23 -07:00
2009-03-27 14:43:59 -04:00
2009-03-01 00:19:35 -08:00
2009-01-21 14:34:05 -08:00
2009-04-14 11:11:52 +02:00
2009-03-23 16:25:46 +01:00
2009-04-09 00:27:13 +01:00
2009-05-08 19:22:21 -07:00
2009-01-12 20:56:41 +01:00
2009-03-26 18:14:15 +01:00
2009-01-30 23:45:32 +05:30
2009-04-02 19:55:30 +01:00
2009-03-26 18:14:15 +01:00
2009-03-24 09:42:54 +00:00
2009-03-18 11:53:48 +01:00
2009-04-07 08:31:11 -07:00
2009-05-12 14:41:47 -07:00
2009-04-04 10:43:31 +01:00
2009-03-30 21:46:44 +02:00
2009-01-30 23:46:03 +05:30
2009-01-05 08:40:21 -08:00
2009-01-05 08:40:25 -08:00
2009-01-05 08:40:25 -08:00
2009-04-03 14:53:32 -07:00
2009-04-01 15:42:34 -07:00
2009-04-16 16:17:11 -07:00
2009-01-30 23:46:40 +05:30
2009-01-30 23:47:07 +05:30
2009-04-07 08:12:38 +02:00
2009-01-30 23:47:59 +05:30
2009-01-30 23:48:33 +05:30
2009-02-10 00:41:26 +01:00
2009-02-15 22:43:33 -08:00
2009-05-05 12:26:24 -07:00
2009-04-29 17:32:40 -07:00
2009-04-01 08:59:20 -07:00
2009-03-16 08:32:27 -06:00
2009-01-08 08:31:00 -08:00
2009-01-08 08:31:00 -08:00
2009-01-08 17:14:59 -08:00
2009-04-03 11:10:33 -07:00
2008-12-29 11:27:46 +02:00
2009-04-13 15:04:29 -07:00
2008-11-20 20:29:48 -08:00
2009-04-03 12:13:03 +02:00
2009-04-13 15:04:30 -07:00
2008-11-20 00:49:27 -08:00
2009-03-24 20:56:50 +01:00
2009-02-18 15:37:56 -08:00
2008-10-30 11:38:45 -07:00
2009-04-27 02:53:51 -07:00
2009-03-31 23:00:26 -04:00
2009-05-09 10:51:34 -04:00
2009-04-03 16:42:37 +01:00
2009-04-03 16:42:39 +01:00
2009-04-22 13:02:09 +10:00
2009-02-07 20:03:33 -05:00
2009-04-09 05:43:32 +02:00
2008-12-01 19:14:02 +01:00
2008-11-24 20:00:40 +01:00
2009-03-20 01:33:32 -07:00
2009-01-02 10:19:38 -08:00
2009-01-30 23:49:39 +05:30
2009-04-22 08:35:10 +02:00
2009-03-13 14:35:31 +01:00
2009-01-30 23:50:12 +05:30
2009-04-06 01:41:22 +02:00
2009-01-21 14:03:37 -08:00
2009-02-17 17:37:40 -08:00
2009-04-01 21:42:26 +02:00
2009-03-30 15:14:53 +02:00
2009-01-30 23:51:17 +05:30
2009-01-04 01:00:51 +01:00
2009-04-03 09:48:29 -07:00
2008-11-20 20:32:15 -08:00
2009-03-31 14:52:52 +02:00
2009-02-10 14:56:59 -08:00
2009-03-28 21:34:44 +01:00
2009-02-24 19:19:49 +01:00
2009-04-06 18:11:41 -07:00
2009-03-28 21:34:45 +01:00
2009-04-13 17:02:14 +02:00
2009-01-21 14:02:26 -08:00
2009-01-30 23:51:41 +05:30
2009-04-08 14:13:03 +02:00
2009-04-02 19:04:53 -07:00
2009-05-11 15:24:07 -04:00
2009-01-30 23:52:16 +05:30
2009-01-30 23:52:41 +05:30
2009-03-26 18:14:15 +01:00
2008-12-17 15:47:48 -08:00
2009-03-28 13:30:43 -07:00
2009-01-30 23:53:12 +05:30
2009-03-27 00:46:44 -07:00
2009-01-30 23:53:40 +05:30
2009-01-30 23:54:53 +05:30
2009-05-18 22:11:22 -07:00
2009-01-30 23:55:45 +05:30
2009-03-26 18:14:14 +01:00
2009-02-14 22:58:35 -08:00
2009-01-30 23:56:13 +05:30
2009-01-30 23:56:48 +05:30
2009-04-27 03:23:54 -07:00
2009-02-14 22:58:35 -08:00
2009-04-16 02:02:07 -07:00
2009-05-18 22:19:19 -07:00
2009-01-30 23:57:16 +05:30
2009-02-11 15:27:15 +11:00
2009-02-02 21:40:58 +05:30
2008-11-16 19:32:39 -08:00
2009-01-30 23:57:43 +05:30
2009-02-01 01:04:33 -08:00
2009-04-13 15:04:29 -07:00
2009-04-27 19:51:58 -07:00
2008-11-15 12:26:44 -08:00
2009-04-28 09:36:24 -07:00
2009-04-03 21:46:01 +01:00
2009-04-07 14:07:52 -07:00
2009-03-02 09:43:32 +01:00
2009-03-24 09:42:54 +00:00
2009-01-15 16:39:38 -08:00
2009-01-05 22:46:26 -05:00
2009-01-30 23:58:12 +05:30
2009-03-26 18:14:15 +01:00
2009-04-07 08:31:09 -07:00
2009-04-21 13:41:48 -07:00
2009-03-31 01:14:44 +04:00
2009-04-21 13:41:48 -07:00
2009-01-30 23:59:12 +05:30
2009-01-30 23:58:41 +05:30
2009-01-30 23:59:44 +05:30
2009-01-31 00:00:06 +05:30
2009-04-09 10:35:30 -07:00
2009-03-13 01:32:36 +01:00
2009-03-27 17:28:43 +01:00
2009-03-24 12:15:23 +01:00
2009-01-02 10:19:40 -08:00
2009-03-30 12:43:22 -03:00
2009-03-26 18:14:15 +01:00
2009-04-14 07:50:56 -04:00
2009-04-14 10:10:47 -04:00
2009-03-26 18:14:17 +01:00
2008-12-23 15:21:56 -05:00
2009-01-05 08:40:30 -08:00
2009-01-15 16:39:41 -08:00
2009-03-31 13:05:32 +10:30
2009-05-09 10:49:41 -04:00
2009-01-22 10:18:58 +01:00
2009-04-16 16:17:10 -07:00
2008-10-31 10:01:56 +01:00
2009-04-03 12:21:12 +02:00
2008-11-14 10:39:14 +11:00
2009-01-20 20:52:10 -08:00
2009-04-09 07:49:44 -07:00
2009-03-24 16:38:26 -07:00
2009-04-07 08:31:08 -07:00
2009-03-24 11:03:11 +02:00
2009-03-24 11:02:55 +02:00
2009-04-22 13:52:09 +03:00
2009-02-11 10:18:04 +01:00
2009-04-06 16:06:26 +01:00
2009-04-06 16:06:26 +01:00
2009-01-08 12:38:58 +00:00
2009-04-06 16:06:26 +01:00
2008-12-30 09:26:11 +10:30
2009-04-16 15:28:23 -04:00
2008-11-24 11:43:21 -05:00
2008-11-27 12:37:54 +01:00
2008-11-16 19:37:55 -08:00
2009-04-02 19:05:10 -07:00
2009-04-01 08:59:17 -07:00
2009-01-21 10:39:51 +01:00
2009-03-12 12:58:05 -05:00
2008-12-29 04:59:31 -08:00
2009-02-27 16:07:32 +09:00
2008-11-28 16:47:41 +01:00
2009-03-26 18:14:15 +01:00
2008-11-16 18:59:45 -08:00
2009-05-17 21:03:42 -07:00
2009-05-02 15:36:09 -07:00
2009-01-06 15:59:00 -08:00
2009-04-05 11:04:19 -07:00
2009-01-09 16:54:41 -08:00
2009-04-07 08:12:38 +02:00
2009-01-06 15:59:00 -08:00
2009-04-29 17:32:35 -07:00
2009-01-31 00:00:35 +05:30
2009-03-13 15:57:40 -05:00
2009-01-09 22:44:25 +01:00
2009-01-11 17:55:16 +01:00
2009-01-08 08:31:07 -08:00
2009-04-02 19:04:48 -07:00
2009-04-02 19:04:48 -07:00
2009-05-02 15:36:10 -07:00
2009-02-21 00:09:42 +01:00
2009-04-05 10:33:07 -07:00
2009-03-31 23:00:27 -04:00
2009-03-24 16:38:24 -07:00
2009-04-05 11:04:19 -07:00
2009-01-05 08:40:13 +10:30
2009-03-31 13:05:30 +10:30
2009-03-26 10:56:35 -07:00
2009-04-01 07:38:54 -04:00
2009-03-26 18:14:14 +01:00
2009-01-22 13:57:41 -08:00
2009-01-31 00:01:11 +05:30
2009-03-20 10:48:14 -07:00
2009-01-14 18:09:02 +01:00
2009-05-09 10:49:42 -04:00
2009-03-27 14:44:00 -04:00
2009-02-14 22:58:35 -08:00
2009-03-04 00:03:08 -08:00
2009-03-13 12:09:29 -07:00
2009-02-15 22:43:33 -08:00
2009-03-15 19:59:13 -07:00
2009-05-18 15:15:06 -07:00
2009-02-14 22:58:35 -08:00
2009-03-24 16:37:55 -07:00
2009-03-28 23:55:58 -07:00
2009-04-03 17:41:23 -07:00
2009-04-03 16:42:42 +01:00
2009-04-03 16:42:44 +01:00
2009-01-31 00:02:27 +05:30
2009-04-03 16:42:43 +01:00
2008-12-23 15:21:37 -05:00
2009-04-01 13:28:15 -04:00
2009-04-03 17:41:12 -07:00
2009-03-10 20:33:18 -04:00
2009-04-07 08:31:20 -07:00
2009-05-13 15:44:39 -04:00
2009-04-30 15:32:11 +00:00
2009-01-06 15:59:00 -08:00
2009-03-31 23:00:27 -04:00
2009-02-03 19:33:51 +05:30
2009-01-08 16:25:18 +11:00
2008-12-21 14:21:14 +11:00
2009-01-09 15:49:06 -07:00
2009-04-27 02:53:47 -07:00
2009-05-02 15:36:10 -07:00
2009-04-27 02:53:45 -07:00
2009-01-07 22:48:15 +01:00
2009-01-07 09:59:50 -08:00
2009-04-02 19:04:56 -07:00
2009-04-01 08:59:13 -07:00
2009-04-03 16:42:36 +01:00
2009-04-03 16:42:39 +01:00
2009-04-01 08:59:13 -07:00
2009-04-07 08:44:06 -07:00
2009-01-09 03:39:43 -05:00
2009-05-18 21:08:20 -07:00
2009-04-22 15:59:41 -07:00
2009-03-20 10:47:57 -07:00
2009-04-06 11:25:06 -07:00
2009-03-20 10:47:49 -07:00
2009-02-02 20:15:47 +03:00
2009-01-06 17:10:04 -08:00
2009-04-21 19:40:00 -07:00
2009-04-21 19:40:00 -07:00
2009-03-26 18:14:15 +01:00
2009-01-15 16:39:41 -08:00
2009-01-31 00:02:54 +05:30
2009-04-27 02:53:46 -07:00
2009-01-08 08:31:12 -08:00
2009-01-08 08:31:12 -08:00
2009-02-14 22:58:35 -08:00
2009-04-15 12:10:12 +02:00
2009-01-31 00:03:40 +05:30
2009-02-10 17:18:17 -08:00
2009-04-24 08:54:21 +02:00
2009-05-08 19:22:21 -07:00
2008-12-29 09:39:53 -05:00
2009-03-24 16:38:26 -07:00
2009-04-01 08:59:13 -07:00
2009-01-06 15:59:12 -08:00
2008-12-12 17:00:07 +01:00
2009-03-27 22:23:52 -04:00
2009-03-13 16:09:12 -07:00
2009-03-26 18:14:14 +01:00
2009-03-31 01:14:44 +04:00
2009-04-02 19:05:00 -07:00
2009-03-30 15:22:01 +02:00
2009-01-07 21:44:20 +01:00
2009-01-07 21:44:20 +01:00
2009-03-26 02:18:35 +01:00
2009-03-26 02:18:37 +01:00
2009-01-15 16:39:41 -08:00
2009-01-05 18:31:12 -08:00
2009-01-31 00:05:59 +05:30
2008-11-12 17:17:17 -08:00
2009-01-10 06:04:33 -08:00
2009-04-03 12:23:05 +02:00
2008-11-16 19:37:55 -08:00
2008-11-16 19:39:21 -08:00
2009-04-03 12:23:05 +02:00
2009-04-03 12:23:05 +02:00
2009-04-14 11:31:50 +02:00
2009-02-26 23:42:11 -08:00
2009-03-30 14:28:58 -07:00
2009-03-30 12:16:38 -07:00
2009-05-09 10:49:39 -04:00
2009-03-30 12:33:01 -07:00
2009-05-09 10:49:39 -04:00
2009-01-15 16:39:37 -08:00
2008-10-30 11:38:45 -07:00
2009-04-22 16:54:27 -04:00
2009-03-22 18:38:25 +01:00
2009-01-06 15:59:28 -08:00
2009-01-06 15:59:07 -08:00
2009-05-09 10:49:41 -04:00
2009-03-08 16:35:53 -07:00
2009-04-02 19:04:51 -07:00
2009-04-01 08:59:24 -07:00
2009-03-19 23:49:41 -07:00
2009-05-18 22:15:56 -07:00
2009-02-16 00:03:10 -08:00
2009-04-19 10:47:45 -07:00
2009-04-26 09:20:38 -07:00
2008-11-14 10:39:18 +11:00
2009-04-03 12:08:57 +02:00
2009-03-26 18:14:15 +01:00
2009-03-30 22:05:11 +10:30
2009-01-02 10:19:43 -08:00
2009-04-07 08:44:05 -07:00
2009-04-08 14:33:38 -07:00
2009-01-02 10:19:42 -08:00
2009-03-07 18:41:38 -08:00
2009-01-29 16:56:32 +09:00
2009-02-27 16:53:50 +09:00
2009-04-13 15:04:29 -07:00
2009-01-31 00:06:33 +05:30
2009-05-18 22:11:22 -07:00
2009-04-03 12:23:06 +02:00
2009-02-20 17:57:48 -08:00
2008-12-29 15:34:05 +02:00
2009-04-21 19:40:00 -07:00
2009-04-03 12:23:06 +02:00
2009-03-13 11:24:31 +01:00
2009-04-28 01:53:11 -07:00
2009-04-27 02:45:02 -07:00
2009-04-23 04:04:32 -07:00
2009-02-15 22:43:33 -08:00
2009-03-27 12:18:56 -04:00
2009-01-31 00:07:00 +05:30
2009-02-10 00:02:54 +01:00
2009-04-02 19:05:11 -07:00
2009-04-15 12:10:12 +02:00
2008-11-23 11:53:50 +01:00
2009-01-05 08:40:14 +10:30
2009-04-05 10:30:21 -07:00
2009-04-10 15:48:52 +02:00
2009-03-26 18:14:14 +01:00
2009-04-19 20:08:42 +02:00
2009-01-14 19:56:50 -08:00
2009-04-01 08:59:15 -07:00
2009-01-11 04:54:34 +01:00
2009-04-02 19:05:01 -07:00
2009-05-12 14:11:35 -07:00
2009-02-01 01:04:33 -08:00
2009-01-31 00:08:18 +05:30
2009-04-20 02:25:26 -07:00
2008-11-11 09:30:23 -08:00
2009-03-27 16:58:22 -04:00
2008-12-30 09:05:12 +10:30
2009-01-01 10:12:25 +10:30
2009-03-26 18:14:14 +01:00
2009-02-15 22:43:32 -08:00
2009-03-30 17:17:35 -07:00
2009-02-18 15:37:53 -08:00
2009-03-27 12:59:54 +08:00
2009-03-26 18:14:14 +01:00
2009-02-25 18:38:15 +01:00
2009-03-13 14:35:31 +01:00
2009-02-03 11:23:12 +11:00
2009-02-26 18:44:06 +01:00
2009-04-02 19:05:00 -07:00
2009-03-10 14:13:14 -04:00
2009-04-07 08:44:04 -07:00
2009-01-02 10:19:40 -08:00
2009-03-26 18:14:21 +01:00
2009-03-02 11:06:49 +01:00
2009-03-24 16:38:24 -07:00
2009-03-24 16:20:34 -07:00
2009-04-17 10:50:27 -07:00
2009-02-27 16:26:21 -08:00
2009-03-26 18:14:14 +01:00
2008-12-12 13:00:06 +00:00
2009-04-06 21:44:29 -03:00
2009-03-30 12:43:15 -03:00
2008-12-30 09:26:04 +10:30
2009-01-31 00:11:25 +05:30
2009-01-31 00:12:44 +05:30
2009-05-01 15:34:02 -07:00
2008-12-30 09:25:58 +10:30
2008-12-30 09:26:03 +10:30
2009-02-24 21:52:45 +01:00
2009-04-28 02:24:21 -07:00
2009-01-07 10:00:16 -08:00
2009-01-29 16:00:05 -05:00
2008-11-19 14:47:04 +00:00
2009-04-02 19:04:50 -07:00
2009-05-15 11:32:24 +02:00
2009-03-26 18:14:14 +01:00
93f154b594