You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-18 - 03:40
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackaslot8.osadl.org (updated Wed Jun 18, 2025 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
73832279245,22sleep10-21swapper/119:08:531
665122670,2sleep2271rcuc/219:05:242
73622265231,22sleep00-21swapper/019:08:350
72382255222,20sleep30-21swapper/319:07:023
58962630,2sleep122-21ksoftirqd/123:55:191
176162620,0sleep00-21swapper/022:55:160
150762610,0sleep20-21swapper/222:07:092
148552610,1sleep30-21swapper/320:35:193
197032600,0sleep20-21swapper/219:35:132
78282590,0sleep00-21swapper/021:16:280
163332570,0sleep10-21swapper/100:02:401
249192550,1sleep30-21swapper/322:14:423
118292550,0sleep30-21swapper/323:36:183
260952540,0sleep10-21swapper/122:37:511
20122190,0sleep00-21swapper/000:16:160
12172180,0sleep00-21swapper/021:11:590
41712150,0sleep00-21swapper/021:36:510
251392130,0chrt0-21swapper/023:00:200
143022110,0sleep10-21swapper/123:15:191
757799108,1cyclictest12-21ksoftirqd/019:15:010
757799107,2cyclictest12-21ksoftirqd/000:25:000
302202100,2sleep030225-21gltestperf21:55:180
75889995,3cyclictest30654-21snmpd22:08:053
75889994,3cyclictest19982-21systemd-journal21:51:403
75779990,6cyclictest21558-21dump-pmu-power19:40:010
8170280,1sleep30-21swapper/319:10:153
75889986,1cyclictest29846-21diskmemload23:22:163
75889984,3cyclictest508-21dbus-daemon23:01:453
75889984,3cyclictest19982-21systemd-journal00:03:293
75889983,3cyclictest19982-21systemd-journal22:50:043
75889983,3cyclictest19982-21systemd-journal22:48:493
75889983,3cyclictest19982-21systemd-journal19:33:083
75859985,2cyclictest11736-21in:imuxsock22:53:242
75859984,3cyclictest508-21dbus-daemon19:28:332
75859984,3cyclictest19982-21systemd-journal00:19:202
75859983,4cyclictest0-21swapper/223:45:192
75859983,3cyclictest19982-21systemd-journal21:45:062
75859981,6cyclictest15657-21perf23:40:012
75809986,1cyclictest9197-21apt-get23:35:171
75809984,3cyclictest19982-21systemd-journal00:07:511
75809984,2cyclictest19982-21systemd-journal00:15:591
75809983,3cyclictest19982-21systemd-journal22:16:171
75779984,3cyclictest508-21dbus-daemon23:52:370
75779983,3cyclictest19982-21systemd-journal19:53:590
75779983,2cyclictest2476-21munin-run20:10:000
13867280,4sleep012-21ksoftirqd/000:00:220
75889975,1cyclictest28081-21ssh21:30:303
75889974,2cyclictest508-21dbus-daemon19:17:173
75889974,2cyclictest34-21ksoftirqd/321:10:013
75889974,2cyclictest19349-21df_inode20:45:163
75889973,3cyclictest7883-21expr22:25:163
75889973,3cyclictest5095-21cstates23:55:153
75889973,3cyclictest30654-21snmpd22:03:173
75889973,3cyclictest29741-11brltty19:41:543
75889973,3cyclictest21077-21pool00:33:133
75889973,3cyclictest19982-21systemd-journal21:43:443
75889973,3cyclictest19982-21systemd-journal19:51:553
75889973,3cyclictest19982-21systemd-journal00:39:223
75889973,3cyclictest17729-21sed20:40:203
75889973,1cyclictest0-21swapper/321:55:303
75889972,3cyclictest19982-21systemd-journal19:45:013
75889971,3cyclictest19982-21systemd-journal23:16:213
75889970,3cyclictest3479-21sshd00:17:553
75889970,1cyclictest1025-21ssh23:05:403
75859975,1cyclictest28-21ksoftirqd/219:15:012
75859974,2cyclictest19982-21systemd-journal23:30:322
75859974,2cyclictest11472-21sh00:25:002
75859973,3cyclictest27403-21sed00:40:002
75859973,3cyclictest22614-21sed20:50:222
75859973,3cyclictest19982-21systemd-journal20:25:412
75859973,3cyclictest19982-21systemd-journal00:32:152
75859973,3cyclictest11736-21in:imuxsock21:12:012
75859973,1cyclictest0-21swapper/223:03:242
75859972,3cyclictest7661-21lldpd20:02:032
75859972,3cyclictest19982-21systemd-journal19:51:052
75859971,3cyclictest508-21dbus-daemon23:45:012
75859970,4cyclictest9998-21awk22:50:012
75859970,4cyclictest1-21systemd22:00:002
75809974,2cyclictest7219-21cron20:19:591
75809974,2cyclictest30654-21snmpd23:23:111
75809974,2cyclictest19982-21systemd-journal19:55:001
75809973,3cyclictest483-21apt-get20:05:111
75809973,3cyclictest3070-21cron22:45:011
75809973,3cyclictest19982-21systemd-journal23:27:461
75809973,3cyclictest19982-21systemd-journal23:10:011
75809973,3cyclictest19982-21systemd-journal22:31:341
75809973,3cyclictest19982-21systemd-journal21:47:051
75809973,3cyclictest18206-21sed00:30:201
75809973,3cyclictest0-21swapper/123:30:171
75809973,2cyclictest19982-21systemd-journal23:44:561
75809973,1cyclictest0-21swapper/123:47:221
75809973,1cyclictest0-21swapper/122:49:441
75809973,1cyclictest0-21swapper/122:13:591
75809973,1cyclictest0-21swapper/122:04:191
75809973,1cyclictest0-21swapper/121:38:101
75809973,1cyclictest0-21swapper/121:14:431
75809971,3cyclictest11464-21cron00:25:001
75779973,3cyclictest508-21dbus-daemon21:08:150
75779973,3cyclictest30654-21snmpd19:45:030
75779973,3cyclictest30654-21snmpd00:09:570
75779973,3cyclictest19982-21systemd-journal22:47:340
75779973,3cyclictest0-21swapper/022:36:260
75779973,3cyclictest0-21swapper/021:24:450
75779973,1cyclictest0-21swapper/023:49:330
75779973,1cyclictest0-21swapper/000:33:030
75779973,1cyclictest0-21swapper/000:25:430
75779972,3cyclictest19982-21systemd-journal23:23:260
21592270,0sleep20-21swapper/223:20:242
75889964,1cyclictest30654-21snmpd21:46:083
75889963,2cyclictest7400-21ssh21:16:053
75889963,2cyclictest30993-21ssh22:40:443
75889963,2cyclictest30654-21snmpd23:52:093
75889963,2cyclictest30654-21snmpd23:41:103
75889963,2cyclictest30654-21snmpd00:20:573
75889963,2cyclictest29846-21diskmemload00:27:443
75889963,2cyclictest27514-21ssh23:25:143
75889963,2cyclictest21345-21grep20:50:143
75889963,2cyclictest20808-21wc19:35:183
75889963,2cyclictest19982-21systemd-journal23:10:303
75889963,2cyclictest17229-21ssh22:55:073
75889963,2cyclictest1522-21ssh22:20:193
75889963,2cyclictest1-21systemd23:30:223
75889962,3cyclictest30654-21snmpd21:25:063
75889962,3cyclictest30654-21snmpd21:20:333
75889962,3cyclictest30654-21snmpd21:10:033
75889962,3cyclictest30654-21snmpd21:01:203
75889962,3cyclictest30654-21snmpd20:18:313
75889962,3cyclictest30654-21snmpd19:55:493
75889962,3cyclictest26528-21cat00:10:203
75889962,3cyclictest19982-21systemd-journal00:06:443
75889962,3cyclictest15313-21cat19:25:153
75889962,3cyclictest14933-21expr22:30:153
75889962,3cyclictest1-21systemd20:31:333
75889962,2cyclictest19982-21systemd-journal20:09:353
75889962,1cyclictest0-21swapper/322:35:163
75889962,1cyclictest0-21swapper/321:35:313
75889961,4cyclictest29846-21diskmemload23:48:533
75889961,3cyclictest19982-21systemd-journal20:20:173
75889961,3cyclictest19982-21systemd-journal20:02:463
75889961,1cyclictest9746-21apt-get20:25:133
75889960,6cyclictest0-21swapper/322:19:493
75889960,6cyclictest0-21swapper/320:10:023
75889960,2cyclictest23841-21fschecks_count20:55:163
75859966,0cyclictest0-21swapper/220:40:152
75859964,2cyclictest28-21ksoftirqd/221:05:012
75859964,1cyclictest28-21ksoftirqd/220:50:012
75859963,2cyclictest907-21ssh21:35:132
75859963,2cyclictest31867-21rm23:27:502
75859963,2cyclictest30654-21snmpd23:06:062
75859963,2cyclictest29741-11brltty22:20:022
75859963,2cyclictest28935-21ssh21:31:192
75859963,2cyclictest28399-21ssh00:11:342
75859963,2cyclictest20427-21sh22:56:382
75859963,2cyclictest16756-21ssh21:23:102
75859963,2cyclictest16379-21ssh23:16:442
75859963,2cyclictest11736-21in:imuxsock21:40:482
75859963,2cyclictest10851-21ssh22:26:512
75859962,3cyclictest7661-21lldpd23:52:412
75859962,3cyclictest30654-21snmpd22:36:152
75859962,3cyclictest30654-21snmpd21:50:132
75859962,3cyclictest30654-21snmpd20:57:502
75859962,3cyclictest30654-21snmpd00:05:482
75859962,3cyclictest29468-21grep22:40:172
75859962,3cyclictest29425-21ntpq19:55:182
75859962,3cyclictest18934-21seq21:25:012
75859962,3cyclictest1442-21ntpq20:05:172
75859962,3cyclictest13149-21tr20:30:192
75859962,3cyclictest1-21systemd23:57:392
75859962,2cyclictest7661-21lldpd20:37:042
75859962,1cyclictest0-21swapper/222:30:322
75859962,1cyclictest0-21swapper/222:16:432
75859962,1cyclictest0-21swapper/222:10:182
75859962,1cyclictest0-21swapper/222:00:032
75859962,1cyclictest0-21swapper/200:00:322
75859961,4cyclictest0-21swapper/200:28:332
75859961,3cyclictest19982-21systemd-journal23:10:332
75859961,3cyclictest19982-21systemd-journal21:15:462
75859961,3cyclictest19982-21systemd-journal19:41:042
75859961,3cyclictest19982-21systemd-journal19:13:572
75859961,1cyclictest18984-21kworker/2:221:05:132
75859960,6cyclictest0-21swapper/220:15:362
75859960,1cyclictest7493-21/usr/sbin/munin20:20:172
75809966,0cyclictest3218-21ssh21:59:261
75809963,2cyclictest508-21dbus-daemon21:08:161
75809963,2cyclictest508-21dbus-daemon19:40:391
75809963,2cyclictest31233-21grep20:00:181
75809963,2cyclictest30654-21snmpd23:01:491
75809963,2cyclictest28773-21expr19:55:141
75809963,2cyclictest25058-21sh21:51:111
75809963,2cyclictest20849-21ssh22:57:001
75809963,2cyclictest12462-21ntp_kernel_pll_22:05:181
75809963,2cyclictest1195-21cat22:20:171
75809963,2cyclictest11736-21in:imuxsock19:48:101
75809963,2cyclictest10448-21ssh22:26:281
75809962,3cyclictest7661-21lldpd22:51:401
75809962,3cyclictest30654-21snmpd21:31:111
75809962,3cyclictest30654-21snmpd20:40:291
75809962,3cyclictest30654-21snmpd20:12:421
75809962,3cyclictest30654-21snmpd19:37:511
75809962,3cyclictest30654-21snmpd19:12:381
75809962,3cyclictest26280-21head00:10:171
75809962,3cyclictest25598-21latency_hist21:00:011
75809962,3cyclictest21445-21expr20:50:151
75809962,3cyclictest19982-21systemd-journal21:41:261
75809962,3cyclictest13513-21basename19:20:161
75809962,3cyclictest10515-21cat20:25:171
75809962,1cyclictest0-21swapper/121:16:541
75809962,1cyclictest0-21swapper/100:26:141
75809961,3cyclictest19982-21systemd-journal21:21:111
75809961,3cyclictest19982-21systemd-journal20:31:581
75809961,3cyclictest19982-21systemd-journal20:23:121
75809961,3cyclictest19982-21systemd-journal19:15:121
75809961,3cyclictest19982-21systemd-journal00:36:521
75809960,4cyclictest19982-21systemd-journal23:10:551
75809960,2cyclictest25621-21/usr/sbin/munin21:00:131
75809960,0cyclictest0-21swapper/123:50:121
75779966,0cyclictest0-21swapper/022:01:020
75779965,1cyclictest12-21ksoftirqd/023:25:010
75779963,2cyclictest508-21dbus-daemon00:37:420
75779963,2cyclictest4981-21cat23:55:140
75779963,2cyclictest32059-21head23:05:170
75779963,2cyclictest26337-21irqstats21:00:170
75779963,2cyclictest26222-21uniq00:10:180
75779963,2cyclictest26114-21ssh21:30:060
75779963,2cyclictest15439-21cat19:25:160
75779963,2cyclictest12-21ksoftirqd/023:40:170
75779963,2cyclictest1-21systemd20:20:000
75779963,1cyclictest29846-21diskmemload22:20:080
75779962,3cyclictest3300-21expr20:10:140
75779962,3cyclictest30654-21snmpd22:52:160
75779962,3cyclictest30654-21snmpd22:34:110
75779962,3cyclictest30654-21snmpd20:33:080
75779962,3cyclictest30654-21snmpd20:02:010
75779962,3cyclictest30654-21snmpd19:31:060
75779962,3cyclictest26449-21cat22:15:160
75779962,3cyclictest2510-21expr23:30:180
75779962,3cyclictest23513-21ls20:55:160
75779962,3cyclictest21949-21sed20:50:180
75779962,3cyclictest19566-21which20:45:180
75779962,3cyclictest15611-21expr21:45:140
75779962,3cyclictest14516-21cat20:35:170
75779962,3cyclictest0-21swapper/023:37:450
75779962,3cyclictest0-21swapper/023:13:500
75779962,3cyclictest0-21swapper/022:25:410
75779962,3cyclictest0-21swapper/021:50:450
75779962,1cyclictest0-21swapper/021:40:310
75779962,1cyclictest0-21swapper/021:25:260
75779961,3cyclictest23769-21turbostat19:45:000
75779961,2cyclictest15129-21ssh23:15:320
75779960,5cyclictest19065-21diskstats22:10:150
75779960,3cyclictest29459-21/usr/sbin/munin22:40:160
75779960,3cyclictest14368-21rm22:06:280
75889951,3cyclictest12901-21grep19:20:143
75859952,2cyclictest24420-21forks19:45:172
75859952,2cyclictest17550-21expr19:30:162
75859951,3cyclictest2987-21gdbus20:10:012
75859950,4cyclictest12893-21apt-get19:20:132
75809952,2cyclictest20266-21cat20:45:231
75809952,2cyclictest15780-21cat19:25:181
75809952,2cyclictest14340-21cut20:35:171
75809951,3cyclictest19172-21/usr/sbin/munin21:25:141
75809951,2cyclictest19982-21systemd-journal19:30:131
75779952,2cyclictest7508-21acpi20:20:130
75779952,2cyclictest30654-21snmpd19:20:090
75779952,2cyclictest29128-21/usr/sbin/munin19:55:150
75779952,2cyclictest10417-21cat20:25:160
75779951,3cyclictest16879-21grep20:40:160
75779950,1cyclictest10337-21memory19:15:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional

 
OSZAR »