Frederic Weisbecker 76e1d9047e
perf: Store active software events in a hashlist ...
Each time a software event triggers, we need to walk through
the entire list of events from the current cpu and task contexts
to retrieve a running perf event that matches.
We also need to check a matching perf event is actually counting.
This walk is wasteful and makes the event fast path scaling
down with a growing number of events running on the same
contexts.
To solve this, we store the running perf events in a hashlist to
get an immediate access to them against their type:event_id when
they trigger.
v2: - Fix SWEVENT_HLIST_SIZE definition (and re-learn some basic
maths along the way)
- Only allocate hlist for online cpus, but keep track of the
refcount on offline possible cpus too, so that we allocate it
if needed when it becomes online.
- Drop the kref use as it's not adapted to our tricks anymore.
v3: - Fix bad refcount check (address instead of value). Thanks to
Eric Dumazet who spotted this.
- While exiting cpu, move the hlist release out of the IPI path
to lock the hlist mutex sanely.
Signed-off-by: Frederic Weisbecker <fweisbec@gmail.com >
Cc: Peter Zijlstra <a.p.zijlstra@chello.nl >
Cc: Arnaldo Carvalho de Melo <acme@redhat.com >
Cc: Paul Mackerras <paulus@samba.org >
Cc: Ingo Molnar <mingo@elte.hu >
2010-04-14 18:20:33 +02:00
2010-03-29 17:33:33 +01:00
2010-02-26 05:22:03 -08:00
2010-03-12 15:52:44 -08:00
2010-03-07 22:17:32 +01:00
2010-02-10 23:04:18 -08:00
2010-02-16 16:01:22 -08:00
2010-03-07 22:17:39 +01:00
2010-03-24 16:31:21 -07:00
2010-02-02 13:02:16 +09:00
2010-03-20 14:29:03 -07:00
2010-02-24 18:32:59 +01:00
2010-01-18 08:25:47 +01:00
2010-02-24 18:34:48 +01:00
2010-02-24 18:34:48 +01:00
2009-12-15 15:01:46 -05:00
2009-12-14 12:49:47 +11:00
2010-03-03 14:49:24 +00:00
2010-03-30 22:02:32 +09:00
2010-03-22 05:32:44 -04:00
2010-03-30 22:02:32 +09:00
2010-03-30 22:02:32 +09:00
2010-03-12 15:53:10 -08:00
2010-01-06 16:18:15 -05:00
2009-12-16 07:20:13 -08:00
2010-03-01 15:11:32 -05:00
2009-12-15 08:53:35 -08:00
2010-02-08 14:38:36 -05:00
2010-03-16 19:47:54 +00:00
2010-03-06 11:26:46 -08:00
2009-12-16 07:20:18 -08:00
2010-04-07 08:38:01 -07:00
2010-02-26 13:58:08 +01:00
2010-02-17 11:17:38 +09:00
2010-02-12 09:41:59 -08:00
2010-02-17 17:27:40 -08:00
2010-02-22 13:44:45 +01:00
2010-02-22 13:44:45 +01:00
2010-03-12 15:52:37 -08:00
2010-03-24 16:31:22 -07:00
2010-03-12 19:10:29 +01:00
2010-02-05 14:54:10 +01:00
2010-03-12 15:52:32 -08:00
2010-02-05 07:35:05 -08:00
2010-02-02 15:58:48 -08:00
2010-03-12 15:52:32 -08:00
2009-12-09 17:09:36 +11:00
2010-03-06 11:26:29 -08:00
2010-03-04 11:46:14 +01:00
2009-12-15 08:53:28 -08:00
2009-12-15 08:53:32 -08:00
2010-02-04 13:29:53 -08:00
2010-03-30 22:02:32 +09:00
2010-03-06 02:32:27 +00:00
2010-03-07 17:04:52 -08:00
2010-02-05 12:22:34 +01:00
2009-12-10 23:52:01 +00:00
2010-03-06 02:32:33 +00:00
2010-03-06 02:32:31 +00:00
2009-12-10 23:52:05 +00:00
2010-03-12 15:52:42 -08:00
2010-03-04 08:20:14 -08:00
2009-12-08 19:26:55 -08:00
2010-01-12 10:02:46 +01:00
2010-01-12 10:14:42 +01:00
2009-12-15 08:53:25 -08:00
2010-02-26 08:25:35 +01:00
2009-12-15 08:53:33 -08:00
2010-03-06 11:26:46 -08:00
2010-03-06 11:26:46 -08:00
2009-12-10 08:54:14 -06:00
2009-12-15 08:53:27 -08:00
2010-03-05 14:00:17 -08:00
2010-01-25 12:26:38 -02:00
2010-02-20 13:13:44 -08:00
2009-12-10 15:02:51 +01:00
2010-03-29 14:30:19 -07:00
2009-12-23 13:44:12 +01:00
2010-03-29 14:30:19 -07:00
2009-12-23 13:33:54 +01:00
2010-02-26 19:19:39 +02:00
2010-03-01 16:20:37 +10:00
2010-02-25 10:34:49 +01:00
2009-12-17 20:55:57 -08:00
2009-12-16 12:16:42 -05:00
2010-02-24 20:36:55 +01:00
2009-12-29 19:58:17 +01:00
2010-03-06 11:26:25 -08:00
2009-12-11 11:24:52 -08:00
2010-03-26 23:51:44 +01:00
2009-12-17 10:58:17 -05:00
2010-04-07 08:38:04 -07:00
2010-03-29 09:14:47 -07:00
2010-03-30 22:02:32 +09:00
2010-03-10 14:47:18 +01:00
2010-03-26 11:33:55 +01:00
2010-03-30 22:02:32 +09:00
2009-12-16 12:16:49 -05:00
2010-02-17 11:17:38 +09:00
2010-03-06 11:26:27 -08:00
2010-03-01 14:08:10 +00:00
2009-12-08 20:30:41 -08:00
2010-02-04 00:50:44 -08:00
2009-12-16 07:20:01 -08:00
2010-02-25 17:39:16 +01:00
2010-02-26 17:05:10 -08:00
2010-02-09 11:13:56 +01:00
2009-12-14 23:55:34 +01:00
2010-03-07 22:17:09 +01:00
2009-12-15 08:53:12 -08:00
2010-02-27 16:23:39 +01:00
2010-03-13 20:56:56 +01:00
2010-02-12 17:32:41 +00:00
2010-03-02 12:23:42 +01:00
2010-03-14 11:14:58 -07:00
2010-03-02 12:23:49 +01:00
2010-02-26 13:58:08 +01:00
2009-12-11 23:55:42 -08:00
2010-02-18 14:30:17 -08:00
2010-01-19 01:52:36 -08:00
2010-01-22 16:11:27 -05:00
2009-12-08 20:42:28 -08:00
2010-02-12 16:56:07 -08:00
2010-02-18 14:08:38 -08:00
2010-02-04 20:24:10 -08:00
2010-02-17 16:35:17 -08:00
2010-03-21 21:19:02 -07:00
2010-02-02 07:32:29 -08:00
2010-02-07 03:06:22 -05:00
2009-12-09 19:43:33 -08:00
2010-01-11 16:28:01 -08:00
2010-02-16 15:55:17 -08:00
2010-03-12 15:52:40 -08:00
2009-12-15 20:42:06 +01:00
2010-03-08 23:19:15 -08:00
2010-03-30 22:02:32 +09:00
2010-03-12 15:52:36 -08:00
2009-12-16 07:20:18 -08:00
2010-03-23 13:33:50 -07:00
2010-03-12 15:52:40 -08:00
2010-03-12 15:52:40 -08:00
2010-03-03 08:15:37 -08:00
2010-04-01 16:12:48 -07:00
2009-12-11 15:18:07 -08:00
2010-03-30 22:02:32 +09:00
2010-03-30 22:02:32 +09:00
2009-12-15 08:53:26 -08:00
2010-01-30 01:47:49 -08:00
2010-03-02 10:41:26 -08:00
2009-12-16 12:19:59 +01:00
2010-04-07 08:38:04 -07:00
2010-02-16 16:01:21 -08:00
2010-02-17 11:17:38 +09:00
2010-04-07 08:38:02 -07:00
2010-01-07 11:58:37 -06:00
2010-01-11 09:34:04 -08:00
2010-03-12 15:53:10 -08:00
2009-12-31 19:45:04 +00:00
2010-03-07 17:04:49 -08:00
2010-02-25 17:49:25 +01:00
2009-12-16 06:56:12 -08:00
2010-03-01 12:36:13 -03:00
2010-03-01 12:36:14 -03:00
2009-12-17 11:41:51 +00:00
2009-12-17 11:33:33 +00:00
2009-12-17 11:27:09 +00:00
2010-04-06 10:55:33 -04:00
2009-12-15 08:53:36 -08:00
2010-01-12 21:02:00 -08:00
2010-03-06 11:26:35 -08:00
2009-12-26 20:40:34 -08:00
2010-02-03 17:39:50 +11:00
2010-03-12 15:53:10 -08:00
2010-02-09 11:13:56 +01:00
2010-02-19 03:35:12 -05:00
2010-03-12 15:52:38 -08:00
2009-12-15 08:53:20 -08:00
2010-03-17 18:43:47 -07:00
2009-12-15 08:53:12 -08:00
2009-12-15 08:53:20 -08:00
2010-01-15 01:43:29 -08:00
2010-03-12 15:52:28 -08:00
2010-04-08 13:37:18 +02:00
2009-12-12 13:08:15 +01:00
2010-03-08 16:55:37 +01:00
2010-03-03 14:07:59 -05:00
2010-04-05 19:50:02 -07:00
2010-03-04 08:15:33 -08:00
2010-02-10 23:49:08 +09:00
2009-12-22 12:27:34 -05:00
2010-02-03 20:32:28 -08:00
2010-03-18 21:16:45 -07:00
2010-03-25 11:17:26 +01:00
2010-02-19 08:03:28 +01:00
2010-03-20 14:29:03 -07:00
2010-01-13 20:38:26 -08:00
2010-03-05 15:46:18 -05:00
2010-03-05 15:44:56 -05:00
2009-12-16 10:43:34 -08:00
2009-12-14 18:12:08 -05:00
2010-02-13 12:26:02 +09:00
2010-02-19 15:52:40 -05:00
2009-12-15 08:53:13 -08:00
2010-03-12 15:52:38 -08:00
2009-12-15 08:53:12 -08:00
2010-02-14 07:13:47 -07:00
2010-03-18 07:30:31 -06:00
2010-02-15 15:14:34 +02:00
2009-12-16 07:19:57 -08:00
2010-01-06 19:47:10 +11:00
2010-04-07 08:38:05 -07:00
2009-12-17 07:23:42 -08:00
2010-01-27 09:20:03 -08:00
2010-02-22 16:15:17 -08:00
2010-03-03 08:47:22 -08:00
2010-02-23 23:14:47 -08:00
2010-02-22 16:21:02 -08:00
2010-03-12 15:52:42 -08:00
2010-03-12 15:52:42 -08:00
2010-03-03 07:34:18 -08:00
2010-04-05 09:39:11 -07:00
2010-04-14 18:20:33 +02:00
2010-01-17 21:52:11 +11:00
2010-02-04 10:23:02 -08:00
2010-02-24 08:30:08 +01:00
2010-03-07 17:04:49 -08:00
2010-01-13 10:51:39 +01:00
2010-03-06 21:28:37 +01:00
2010-03-06 21:28:37 +01:00
2009-12-15 17:35:26 -05:00
2010-01-11 09:45:37 -08:00
2010-03-12 15:53:11 -08:00
2010-01-15 16:59:21 -05:00
2010-03-26 11:33:55 +01:00
2009-12-17 11:46:01 +00:00
2010-03-05 00:20:31 +01:00
2010-03-05 00:20:30 +01:00
2010-02-08 18:19:41 -06:00
2010-02-10 17:47:17 -08:00
2010-02-25 11:45:20 +01:00
2010-02-25 09:41:02 +01:00
2010-02-28 19:23:06 -08:00
2010-03-18 21:25:32 +01:00
2010-02-26 08:20:46 +01:00
2010-02-26 08:20:46 +01:00
2010-03-12 15:52:44 -08:00
2010-03-05 13:25:52 -05:00
2010-03-24 16:31:21 -07:00
2010-01-04 11:33:58 +01:00
2010-02-26 20:39:09 +01:00
2010-03-02 14:28:49 -05:00
2010-03-06 11:26:27 -08:00
2010-03-12 15:52:28 -08:00
2010-03-12 15:53:10 -08:00
2010-02-28 19:23:06 -08:00
2009-12-14 23:55:33 +01:00
2009-12-14 23:55:32 +01:00
2010-03-13 01:21:21 +01:00
2009-12-15 08:53:26 -08:00
2010-03-26 11:33:55 +01:00
2010-03-30 22:02:32 +09:00
2009-12-16 07:20:10 -08:00
2010-02-22 15:45:54 -08:00
2010-03-12 15:52:43 -08:00
2010-03-10 11:55:05 +09:00
2010-01-21 23:55:25 -08:00
2010-02-16 13:38:56 +09:00
2009-12-16 12:16:49 -05:00
2010-03-12 15:52:44 -08:00
2010-03-24 11:13:35 -07:00
2009-12-11 09:17:02 +01:00
2010-02-26 19:19:39 +02:00
2009-12-20 10:17:59 +02:00
2009-12-16 07:20:04 -08:00
2010-03-06 11:26:32 -08:00
2010-03-08 10:45:27 -08:00
2010-03-27 08:29:01 -07:00
2009-12-16 22:32:29 -05:00
2009-12-14 23:55:33 +01:00
2009-12-14 23:55:33 +01:00
2009-12-14 23:55:32 +01:00
2009-12-14 23:55:32 +01:00
2009-12-14 23:55:32 +01:00
2010-03-13 01:21:21 +01:00
2010-03-03 07:34:18 -08:00
2010-01-07 17:06:06 -08:00
2010-01-14 22:38:09 -05:00
2010-03-12 15:52:36 -08:00
2010-03-22 13:12:33 -07:00
2010-02-16 15:55:18 -08:00
2010-03-07 17:04:47 -08:00
2010-03-07 17:04:52 -08:00
2010-02-04 14:20:41 +11:00
2010-03-30 22:02:32 +09:00
2010-02-18 15:43:09 -08:00
2009-12-16 07:20:00 -08:00
2010-01-29 10:15:19 +01:00
2010-01-21 13:39:03 +01:00
2009-12-09 14:05:49 -05:00
2009-12-16 07:20:08 -08:00
2010-03-16 10:10:40 +01:00
2010-03-02 14:55:11 -08:00
2010-03-12 15:52:43 -08:00
2010-03-19 07:17:57 -07:00
2010-01-07 11:58:36 -06:00
2009-12-11 11:55:26 -08:00
2010-03-19 07:24:23 -07:00
2009-12-10 22:55:37 +01:00
2009-12-12 13:08:15 +01:00
2010-03-01 22:21:58 +11:00
2010-01-15 01:43:29 -08:00
2010-02-26 15:10:40 -03:00
2010-03-13 08:57:28 -06:00
2010-02-24 14:22:08 +10:30
2010-02-24 14:22:26 +10:30
2010-04-08 09:46:19 +09:30
2010-03-13 08:57:29 -06:00
2010-01-05 09:17:33 +09:00
2010-03-19 07:17:52 -07:00
2009-12-23 07:57:07 -05:00
2009-12-16 12:16:49 -05:00
2010-02-22 16:19:45 -08:00
76e1d9047e