From Joshua Young, 7 Years ago, written in Plain Text.
- go back
Embed
Viewing differences between and gdb 1536
  1. root@freeswitch-test:/var/www/html/fs# gdb /usr/local/freeswitch/bin/freeswitch core.1536
  2. GNU gdb (Debian 7.7.1+dfsg-5) 7.7.1
  3. Copyright (C) 2014 Free Software Foundation, Inc.
  4. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
  5. This is free software: you are free to change and redistribute it.
  6. There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
  7. and "show warranty" for details.
  8. This GDB was configured as "x86_64-linux-gnu".
  9. Type "show configuration" for configuration details.
  10. For bug reporting instructions, please see:
  11. <http://www.gnu.org/software/gdb/bugs/>.
  12. Find the GDB manual and other documentation resources online at:
  13. <http://www.gnu.org/software/gdb/documentation/>.
  14. For help, type "help".
  15. Type "apropos word" to search for commands related to "word"...
  16. Reading symbols from /usr/local/freeswitch/bin/freeswitch...done.
  17. [New LWP 1537]
  18. [New LWP 1538]
  19. [New LWP 1539]
  20. [New LWP 1540]
  21. [New LWP 1541]
  22. [New LWP 1542]
  23. [New LWP 1543]
  24. [New LWP 1545]
  25. [New LWP 1546]
  26. [New LWP 1547]
  27. [New LWP 1548]
  28. [New LWP 1549]
  29. [New LWP 1550]
  30. [New LWP 1551]
  31. [New LWP 1552]
  32. [New LWP 1553]
  33. [New LWP 1554]
  34. [New LWP 1555]
  35. [New LWP 1556]
  36. [New LWP 1557]
  37. [New LWP 1558]
  38. [New LWP 1559]
  39. [New LWP 1561]
  40. [New LWP 1560]
  41. [New LWP 1562]
  42. [New LWP 1563]
  43. [New LWP 1564]
  44. [New LWP 1565]
  45. [New LWP 1566]
  46. [New LWP 1567]
  47. [New LWP 1568]
  48. [New LWP 1569]
  49. [New LWP 1570]
  50. [New LWP 1571]
  51. [New LWP 1572]
  52. [New LWP 1573]
  53. [New LWP 1574]
  54. [New LWP 1575]
  55. [New LWP 1576]
  56. [New LWP 1577]
  57. [New LWP 1578]
  58. [New LWP 1584]
  59. [New LWP 1587]
  60. [New LWP 1588]
  61. [New LWP 1591]
  62. [New LWP 1536]
  63. [Thread debugging using libthread_db enabled]
  64. Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
  65. Core was generated by `/usr/local/freeswitch/bin/freeswitch'.
  66. #0  0x00007fb193bfd893 in select () at ../sysdeps/unix/syscall-template.S:81
  67. 81      ../sysdeps/unix/syscall-template.S: No such file or directory.
  68. (gdb) thread apply all bt
  69.  
  70. Thread 46 (Thread 0x7fb195d517c0 (LWP 1536)):
  71. #0  0x00007fb193bfd893 in select () at ../sysdeps/unix/syscall-template.S:81
  72. #1  0x00007fb195304565 in apr_sleep (t=<optimized out>) at time/unix/time.c:246
  73. #2  0x00007fb1950d38fc in do_sleep (t=<optimized out>) at src/switch_time.c:164
  74. #3  0x00007fb195020442 in switch_core_runtime_loop (bg=<optimized out>) at src/switch_core.c:1191
  75. #4  0x0000000000402e24 in main (argc=7, argv=0x7fffdb1798d8) at src/switch.c:1208
  76.  
  77. Thread 45 (Thread 0x7fb111002700 (LWP 1591)):
  78. #0  pthread_rwlock_rdlock () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_rwlock_rdlock.S:87
  79. #1  0x00007fb1952fcd09 in apr_thread_rwlock_rdlock (rwlock=<optimized out>) at locks/unix/thread_rwlock.c:68
  80. #2  0x00007fb194fe2be5 in switch_thread_rwlock_rdlock (rwlock=<optimized out>) at src/switch_apr.c:235
  81. #3  0x00007fb194ff9557 in video_bug_thread (thread=<optimized out>, obj=0x7fb0d406b8b8)
  82.     at src/switch_core_media_bug.c:684
  83. #4  0x00007fb195303000 in dummy_worker (opaque=0x7fb0d00129c0) at threadproc/unix/thread.c:151
  84. #5  0x00007fb19452c064 in start_thread (arg=0x7fb111002700) at pthread_create.c:309
  85. #6  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  86.  
  87. Thread 44 (Thread 0x7fb111422700 (LWP 1588)):
  88. #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  89. #1  0x00007fb1952fcfad in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>)
  90.     at locks/unix/thread_cond.c:68
  91. #2  0x00007fb1952f4ab1 in apr_queue_pop (queue=queue@entry=0x7fb18df4a810, data=data@entry=0x7fb111421dc8)
  92.     at misc/apr_queue.c:276
  93. #3  0x00007fb194fe3af5 in switch_queue_pop (queue=queue@entry=0x7fb18df4a810, data=data@entry=0x7fb111421dc8)
  94.     at src/switch_apr.c:1159
  95. #4  0x00007fb19506e8c9 in switch_event_channel_deliver_thread (thread=<optimized out>, obj=0x7fb18df4a810)
  96.     at src/switch_event.c:2929
  97. #5  0x00007fb19500e1ad in switch_core_session_thread_pool_worker (thread=0x7fb178003bb0, obj=0x80)
  98.     at src/switch_core_session.c:1758
  99. #6  0x00007fb195303000 in dummy_worker (opaque=0x7fb178003bb0) at threadproc/unix/thread.c:151
  100. #7  0x00007fb19452c064 in start_thread (arg=0x7fb111422700) at pthread_create.c:309
  101. #8  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  102.  
  103. Thread 43 (Thread 0x7fb1114ae700 (LWP 1587)):
  104. #0  0x00007fb19452d49b in pthread_join (threadid=140398471161600,
  105.     thread_return=thread_return@entry=0x7fb1114adad8) at pthread_join.c:92
  106. #1  0x00007fb19530327d in apr_thread_join (retval=retval@entry=0x7fb1114adb08, thd=0x7fb0d00129c0)
  107.     at threadproc/unix/thread.c:234
  108. #2  0x00007fb194fe3cea in switch_thread_join (retval=retval@entry=0x7fb1114adb08, thd=<optimized out>)
  109.     at src/switch_apr.c:1300
  110. #3  0x00007fb194ffb8a4 in switch_core_media_bug_close (bug=0x7fb1114adb58) at src/switch_core_media_bug.c:1285
  111. #4  0x00007fb194ffbb4b in switch_core_media_bug_remove_all_function (session=0x7fb1580360a8, function=0x0)
  112.     at src/switch_core_media_bug.c:1220
  113. #5  0x00007fb1950166d9 in switch_core_session_hangup_state (session=0x7fb1580360a8, force=SWITCH_TRUE)
  114.     at src/switch_core_state_machine.c:841
  115. #6  0x00007fb1950180cd in switch_core_session_run (session=0x7fb1580360a8) at src/switch_core_state_machine.c:615
  116. #7  0x00007fb1950126ee in switch_core_session_thread (thread=<optimized out>, obj=0x7fb1580360a8)
  117.     at src/switch_core_session.c:1695
  118. #8  0x00007fb19500e1ad in switch_core_session_thread_pool_worker (thread=0x7fb158057340, obj=0x0)
  119.     at src/switch_core_session.c:1758
  120. #9  0x00007fb195303000 in dummy_worker (opaque=0x7fb158057340) at threadproc/unix/thread.c:151
  121. #10 0x00007fb19452c064 in start_thread (arg=0x7fb1114ae700) at pthread_create.c:309
  122. ---Type <return> to continue, or q <return> to quit---
  123. #11 0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  124.  
  125. Thread 42 (Thread 0x7fb1114ea700 (LWP 1584)):
  126. #0  __clock_nanosleep (clock_id=1, flags=0, req=0x7fb1114e9d80, rem=0xffffffffffffffff)
  127.     at ../sysdeps/unix/sysv/linux/clock_nanosleep.c:49
  128. #1  0x00007fb1950d395b in do_sleep (t=<optimized out>) at src/switch_time.c:173
  129. #2  0x00007fb14ded728a in vm_event_thread_run (thread=0x1, obj=0x0) at mod_voicemail.c:4109
  130. #3  0x00007fb195303000 in dummy_worker (opaque=0xbf2758) at threadproc/unix/thread.c:151
  131. #4  0x00007fb19452c064 in start_thread (arg=0x7fb1114ea700) at pthread_create.c:309
  132. #5  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  133.  
  134. Thread 41 (Thread 0x7fb111770700 (LWP 1578)):
  135. #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  136. #1  0x00007fb1952fcfad in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>)
  137.     at locks/unix/thread_cond.c:68
  138. #2  0x00007fb1952f4ab1 in apr_queue_pop (queue=queue@entry=0xc7f030, data=data@entry=0x7fb11176fe40)
  139.     at misc/apr_queue.c:276
  140. #3  0x00007fb194fe3af5 in switch_queue_pop (queue=queue@entry=0xc7f030, data=data@entry=0x7fb11176fe40)
  141.     at src/switch_apr.c:1159
  142. #4  0x00007fb195065fee in chat_thread_run (thread=<optimized out>, obj=0xc7f030)
  143.     at src/switch_loadable_module.c:738
  144. #5  0x00007fb195303000 in dummy_worker (opaque=0xc8b020) at threadproc/unix/thread.c:151
  145. #6  0x00007fb19452c064 in start_thread (arg=0x7fb111770700) at pthread_create.c:309
  146. #7  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  147.  
  148. Thread 40 (Thread 0x7fb1117ac700 (LWP 1577)):
  149. #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  150. #1  0x00007fb1952fcfad in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>)
  151.     at locks/unix/thread_cond.c:68
  152. #2  0x00007fb1952f4ab1 in apr_queue_pop (queue=queue@entry=0xa91f88, data=data@entry=0x7fb1117abe40)
  153.     at misc/apr_queue.c:276
  154. #3  0x00007fb194fe3af5 in switch_queue_pop (queue=queue@entry=0xa91f88, data=data@entry=0x7fb1117abe40)
  155.     at src/switch_apr.c:1159
  156. #4  0x00007fb195065fee in chat_thread_run (thread=<optimized out>, obj=0xa91f88)
  157.     at src/switch_loadable_module.c:738
  158. #5  0x00007fb195303000 in dummy_worker (opaque=0xc7f000) at threadproc/unix/thread.c:151
  159. #6  0x00007fb19452c064 in start_thread (arg=0x7fb1117ac700) at pthread_create.c:309
  160. #7  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  161.  
  162. Thread 39 (Thread 0x7fb1117e8700 (LWP 1576)):
  163. #0  0x00007fb194532a9d in read () at ../sysdeps/unix/syscall-template.S:81
  164. #1  0x00007fb1950d5442 in softtimer_runtime () at src/switch_time.c:1168
  165. #2  0x00007fb19505f902 in switch_loadable_module_exec (thread=0xa91f08, obj=0xa91ae8)
  166.     at src/switch_loadable_module.c:113
  167. #3  0x00007fb195303000 in dummy_worker (opaque=0xa91f08) at threadproc/unix/thread.c:151
  168. #4  0x00007fb19452c064 in start_thread (arg=0x7fb1117e8700) at pthread_create.c:309
  169. #5  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  170.  
  171. Thread 38 (Thread 0x7fb111824700 (LWP 1575)):
  172. #0  0x00007fb194532c8d in accept () at ../sysdeps/unix/syscall-template.S:81
  173. #1  0x00007fb195301b26 in apr_socket_accept (new=0x7fb111823df0, sock=0x7fb0f8000b60,
  174.     connection_context=0x7fb0f8004cf8) at network_io/unix/sockets.c:191
  175. #2  0x00007fb18cba87a2 in mod_event_socket_runtime () at mod_event_socket.c:2987
  176. ---Type <return> to continue, or q <return> to quit---
  177. #3  0x00007fb19505f902 in switch_loadable_module_exec (thread=0xa91a50, obj=0xa91630)
  178.     at src/switch_loadable_module.c:113
  179. #4  0x00007fb195303000 in dummy_worker (opaque=0xa91a50) at threadproc/unix/thread.c:151
  180. #5  0x00007fb19452c064 in start_thread (arg=0x7fb111824700) at pthread_create.c:309
  181. #6  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  182.  
  183. Thread 37 (Thread 0x7fb111860700 (LWP 1574)):
  184. #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  185. #1  0x00007fb1952fcfad in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>)
  186.     at locks/unix/thread_cond.c:68
  187. #2  0x00007fb194fe2d95 in switch_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>)
  188.     at src/switch_apr.c:363
  189. #3  0x00007fb14f4ed543 in mod_verto_runtime () at mod_verto.c:6205
  190. #4  0x00007fb19505f902 in switch_loadable_module_exec (thread=0xa91598, obj=0xa91078)
  191.     at src/switch_loadable_module.c:113
  192. #5  0x00007fb195303000 in dummy_worker (opaque=0xa91598) at threadproc/unix/thread.c:151
  193. #6  0x00007fb19452c064 in start_thread (arg=0x7fb111860700) at pthread_create.c:309
  194. #7  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  195.  
  196. Thread 36 (Thread 0x7fb112105700 (LWP 1573)):
  197. #0  0x00007fb194532a9d in read () at ../sysdeps/unix/syscall-template.S:81
  198. #1  0x00007fb1950d4766 in _timerfd_next (timer=0xc60a80) at src/switch_time.c:482
  199. #2  timer_next (timer=0xc60a80) at src/switch_time.c:846
  200. #3  0x00007fb194ffc2d8 in switch_core_timer_next (timer=<optimized out>) at src/switch_core_timer.c:74
  201. #4  0x00007fb11210ab08 in read_stream_thread (thread=0x51, obj=0x7fb112104480) at mod_local_stream.c:449
  202. #5  0x00007fb195303000 in dummy_worker (opaque=0xc614f0) at threadproc/unix/thread.c:151
  203. #6  0x00007fb19452c064 in start_thread (arg=0x7fb112105700) at pthread_create.c:309
  204. #7  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  205.  
  206. Thread 35 (Thread 0x7fb14c076700 (LWP 1572)):
  207. #0  0x00007fb194532a9d in read () at ../sysdeps/unix/syscall-template.S:81
  208. #1  0x00007fb1950d4766 in _timerfd_next (timer=0xc5ca60) at src/switch_time.c:482
  209. #2  timer_next (timer=0xc5ca60) at src/switch_time.c:846
  210. #3  0x00007fb194ffc2d8 in switch_core_timer_next (timer=<optimized out>) at src/switch_core_timer.c:74
  211. #4  0x00007fb11210ab08 in read_stream_thread (thread=0x4f, obj=0x7fb14c075480) at mod_local_stream.c:449
  212. #5  0x00007fb195303000 in dummy_worker (opaque=0xc5d610) at threadproc/unix/thread.c:151
  213. #6  0x00007fb19452c064 in start_thread (arg=0x7fb14c076700) at pthread_create.c:309
  214. #7  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  215.  
  216. Thread 34 (Thread 0x7fb14c0b2700 (LWP 1571)):
  217. #0  0x00007fb194532a9d in read () at ../sysdeps/unix/syscall-template.S:81
  218. #1  0x00007fb1950d4766 in _timerfd_next (timer=0xc58a40) at src/switch_time.c:482
  219. #2  timer_next (timer=0xc58a40) at src/switch_time.c:846
  220. #3  0x00007fb194ffc2d8 in switch_core_timer_next (timer=<optimized out>) at src/switch_core_timer.c:74
  221. #4  0x00007fb11210ab08 in read_stream_thread (thread=0x4d, obj=0x7fb14c0b1480) at mod_local_stream.c:449
  222. #5  0x00007fb195303000 in dummy_worker (opaque=0xc59370) at threadproc/unix/thread.c:151
  223. #6  0x00007fb19452c064 in start_thread (arg=0x7fb14c0b2700) at pthread_create.c:309
  224. #7  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  225.  
  226. Thread 33 (Thread 0x7fb14c0ee700 (LWP 1570)):
  227. #0  0x00007fb194532a9d in read () at ../sysdeps/unix/syscall-template.S:81
  228. #1  0x00007fb1950d4766 in _timerfd_next (timer=0xc54a20) at src/switch_time.c:482
  229. #2  timer_next (timer=0xc54a20) at src/switch_time.c:846
  230. ---Type <return> to continue, or q <return> to quit---
  231. #3  0x00007fb194ffc2d8 in switch_core_timer_next (timer=<optimized out>) at src/switch_core_timer.c:74
  232. #4  0x00007fb11210ab08 in read_stream_thread (thread=0x49, obj=0x7fb14c0ed480) at mod_local_stream.c:449
  233. #5  0x00007fb195303000 in dummy_worker (opaque=0xc55210) at threadproc/unix/thread.c:151
  234. #6  0x00007fb19452c064 in start_thread (arg=0x7fb14c0ee700) at pthread_create.c:309
  235. #7  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  236.  
  237. Thread 32 (Thread 0x7fb14c12a700 (LWP 1569)):
  238. #0  0x00007fb194532a9d in read () at ../sysdeps/unix/syscall-template.S:81
  239. #1  0x00007fb1950d4766 in _timerfd_next (timer=0xc50a00) at src/switch_time.c:482
  240. #2  timer_next (timer=0xc50a00) at src/switch_time.c:846
  241. #3  0x00007fb194ffc2d8 in switch_core_timer_next (timer=<optimized out>) at src/switch_core_timer.c:74
  242. #4  0x00007fb11210ab08 in read_stream_thread (thread=0x4a, obj=0x7fb14c129480) at mod_local_stream.c:449
  243. #5  0x00007fb195303000 in dummy_worker (opaque=0xc511f0) at threadproc/unix/thread.c:151
  244. #6  0x00007fb19452c064 in start_thread (arg=0x7fb14c12a700) at pthread_create.c:309
  245. #7  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  246.  
  247. Thread 31 (Thread 0x7fb14c56a700 (LWP 1568)):
  248. #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  249. #1  0x00007fb1952fcfad in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>)
  250.     at locks/unix/thread_cond.c:68
  251. #2  0x00007fb194fe2d95 in switch_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>)
  252.     at src/switch_apr.c:363
  253. #3  0x00007fb14ce8fd23 in timer_thread_run (thread=<optimized out>, obj=<optimized out>) at mod_spandsp_fax.c:201
  254. #4  0x00007fb195303000 in dummy_worker (opaque=0xc17d90) at threadproc/unix/thread.c:151
  255. #5  0x00007fb19452c064 in start_thread (arg=0x7fb14c56a700) at pthread_create.c:309
  256. #6  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  257.  
  258. Thread 30 (Thread 0x7fb14e33a700 (LWP 1567)):
  259. #0  0x00007fb193bfd893 in select () at ../sysdeps/unix/syscall-template.S:81
  260. #1  0x00007fb195304565 in apr_sleep (t=<optimized out>) at time/unix/time.c:246
  261. #2  0x00007fb1950d38fc in do_sleep (t=<optimized out>) at src/switch_time.c:164
  262. #3  0x00007fb14e5116a2 in node_thread_run (thread=0x0, obj=0x0) at mod_fifo.c:2184
  263. #4  0x00007fb195303000 in dummy_worker (opaque=0xba9fd0) at threadproc/unix/thread.c:151
  264. #5  0x00007fb19452c064 in start_thread (arg=0x7fb14e33a700) at pthread_create.c:309
  265. #6  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  266.  
  267. Thread 29 (Thread 0x7fb14e376700 (LWP 1566)):
  268. #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  269. #1  0x00007fb195006f1e in switch_user_sql_thread (thread=0xbbd13c, obj=0xbbcf90) at src/switch_core_sqldb.c:2135
  270. #2  0x00007fb195303000 in dummy_worker (opaque=0x7fb14e43a6c0) at threadproc/unix/thread.c:151
  271. #3  0x00007fb19452c064 in start_thread (arg=0x7fb14e376700) at pthread_create.c:309
  272. #4  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  273.  
  274. Thread 28 (Thread 0x7fb14ed85700 (LWP 1565)):
  275. #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  276. #1  0x00007fb1952fcfad in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>)
  277.     at locks/unix/thread_cond.c:68
  278. #2  0x00007fb1952f4ab1 in apr_queue_pop (queue=0xae7d68, data=0x7fb14ed84e18) at misc/apr_queue.c:276
  279. #3  0x00007fb18c8416af in sofia_presence_event_thread_run (thread=0xae7e04, obj=0x80) at sofia_presence.c:1624
  280. #4  0x00007fb195303000 in dummy_worker (opaque=0xae8b48) at threadproc/unix/thread.c:151
  281. #5  0x00007fb19452c064 in start_thread (arg=0x7fb14ed85700) at pthread_create.c:309
  282. #6  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  283.  
  284. ---Type <return> to continue, or q <return> to quit---
  285. Thread 27 (Thread 0x7fb14f4be700 (LWP 1564)):
  286. #0  0x00007fb193bfbaed in poll () at ../sysdeps/unix/syscall-template.S:81
  287. #1  0x00007fb19506cbea in switch_wait_socklist (waitlist=0x7fb14f4bdbc0, len=3, ms=100)
  288.     at src/switch_utils.c:2983
  289. #2  0x00007fb14f4ec6e7 in profile_one_loop (profile=0x7fb18c017038) at mod_verto.c:4359
  290. #3  0x00007fb14f4ed210 in runtime (profile=<optimized out>) at mod_verto.c:4487
  291. #4  profile_thread (thread=0x7fb128000a90, obj=0x7fb18c017038) at mod_verto.c:5126
  292. #5  0x00007fb19500e1ad in switch_core_session_thread_pool_worker (thread=0xb48640, obj=0x3)
  293.     at src/switch_core_session.c:1758
  294. #6  0x00007fb195303000 in dummy_worker (opaque=0xb48640) at threadproc/unix/thread.c:151
  295. #7  0x00007fb19452c064 in start_thread (arg=0x7fb14f4be700) at pthread_create.c:309
  296. #8  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  297.  
  298. Thread 26 (Thread 0x7fb14fb47700 (LWP 1563)):
  299. #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
  300. #1  0x00007fb1952fd012 in apr_thread_cond_timedwait (cond=0x7fb18c7bbb68, mutex=0x7fb18c7bbb18,
  301.     timeout=timeout@entry=100000) at locks/unix/thread_cond.c:89
  302. #2  0x00007fb1952f4b9c in apr_queue_pop_timeout (queue=0x7fb18c7bbad8, data=data@entry=0x7fb14fb46e20,
  303.     timeout=timeout@entry=100000) at misc/apr_queue.c:339
  304. #3  0x00007fb194fe3b05 in switch_queue_pop_timeout (queue=<optimized out>, data=data@entry=0x7fb14fb46e20,
  305.     timeout=timeout@entry=100000) at src/switch_apr.c:1164
  306. #4  0x00007fb18c8084b8 in sofia_profile_worker_thread_run (thread=<optimized out>, obj=0xb07860) at sofia.c:2983
  307. #5  0x00007fb195303000 in dummy_worker (opaque=0x7fb18c09cb50) at threadproc/unix/thread.c:151
  308. #6  0x00007fb19452c064 in start_thread (arg=0x7fb14fb47700) at pthread_create.c:309
  309. #7  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  310.  
  311. Thread 25 (Thread 0x7fb14fb83700 (LWP 1562)):
  312. #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  313. #1  0x00007fb195006f1e in switch_user_sql_thread (thread=0x7fb158030184, obj=0x7fb15802ffc0)
  314.     at src/switch_core_sqldb.c:2135
  315. #2  0x00007fb195303000 in dummy_worker (opaque=0x7fb14fc476c0) at threadproc/unix/thread.c:151
  316. #3  0x00007fb19452c064 in start_thread (arg=0x7fb14fb83700) at pthread_create.c:309
  317. #4  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  318.  
  319. Thread 24 (Thread 0x7fb14fcc0700 (LWP 1560)):
  320. #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
  321. #1  0x00007fb1952fd012 in apr_thread_cond_timedwait (cond=0x7fb18c7bbb68, mutex=0x7fb18c7bbb18,
  322.     timeout=timeout@entry=100000) at locks/unix/thread_cond.c:89
  323. #2  0x00007fb1952f4b9c in apr_queue_pop_timeout (queue=0x7fb18c7bbad8, data=data@entry=0x7fb14fcbfe20,
  324.     timeout=timeout@entry=100000) at misc/apr_queue.c:339
  325. #3  0x00007fb194fe3b05 in switch_queue_pop_timeout (queue=<optimized out>, data=data@entry=0x7fb14fcbfe20,
  326.     timeout=timeout@entry=100000) at src/switch_apr.c:1164
  327. #4  0x00007fb18c8084b8 in sofia_profile_worker_thread_run (thread=<optimized out>, obj=0xafa5b0) at sofia.c:2983
  328. #5  0x00007fb195303000 in dummy_worker (opaque=0x7fb18c476f38) at threadproc/unix/thread.c:151
  329. #6  0x00007fb19452c064 in start_thread (arg=0x7fb14fcc0700) at pthread_create.c:309
  330. #7  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  331.  
  332. Thread 23 (Thread 0x7fb14fc84700 (LWP 1561)):
  333. #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
  334. #1  0x00007fb1952fd012 in apr_thread_cond_timedwait (cond=0x7fb18c7bbb68, mutex=0x7fb18c7bbb18,
  335.     timeout=timeout@entry=100000) at locks/unix/thread_cond.c:89
  336. #2  0x00007fb1952f4b9c in apr_queue_pop_timeout (queue=0x7fb18c7bbad8, data=data@entry=0x7fb14fc83e20,
  337.     timeout=timeout@entry=100000) at misc/apr_queue.c:339
  338. ---Type <return> to continue, or q <return> to quit---
  339. #3  0x00007fb194fe3b05 in switch_queue_pop_timeout (queue=<optimized out>, data=data@entry=0x7fb14fc83e20,
  340.     timeout=timeout@entry=100000) at src/switch_apr.c:1164
  341. #4  0x00007fb18c8084b8 in sofia_profile_worker_thread_run (thread=<optimized out>, obj=0xaf3240) at sofia.c:2983
  342. #5  0x00007fb195303000 in dummy_worker (opaque=0x7fb18c4d9b40) at threadproc/unix/thread.c:151
  343. #6  0x00007fb19452c064 in start_thread (arg=0x7fb14fc84700) at pthread_create.c:309
  344. #7  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  345.  
  346. Thread 22 (Thread 0x7fb14fcfc700 (LWP 1559)):
  347. #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
  348. #1  0x00007fb1952fd012 in apr_thread_cond_timedwait (cond=0x7fb18c7bbb68, mutex=0x7fb18c7bbb18,
  349.     timeout=timeout@entry=100000) at locks/unix/thread_cond.c:89
  350. #2  0x00007fb1952f4b9c in apr_queue_pop_timeout (queue=0x7fb18c7bbad8, data=data@entry=0x7fb14fcfbe20,
  351.     timeout=timeout@entry=100000) at misc/apr_queue.c:339
  352. #3  0x00007fb194fe3b05 in switch_queue_pop_timeout (queue=<optimized out>, data=data@entry=0x7fb14fcfbe20,
  353.     timeout=timeout@entry=100000) at src/switch_apr.c:1164
  354. #4  0x00007fb18c8084b8 in sofia_profile_worker_thread_run (thread=<optimized out>, obj=0xb00c60) at sofia.c:2983
  355. #5  0x00007fb195303000 in dummy_worker (opaque=0x7fb18c413b40) at threadproc/unix/thread.c:151
  356. #6  0x00007fb19452c064 in start_thread (arg=0x7fb14fcfc700) at pthread_create.c:309
  357. #7  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  358.  
  359. Thread 21 (Thread 0x7fb14fd38700 (LWP 1558)):
  360. #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  361. #1  0x00007fb195006f1e in switch_user_sql_thread (thread=0x7fb1680299a4, obj=0x7fb1680297d0)
  362.     at src/switch_core_sqldb.c:2135
  363. #2  0x00007fb195303000 in dummy_worker (opaque=0x7fb14ff396c0) at threadproc/unix/thread.c:151
  364. #3  0x00007fb19452c064 in start_thread (arg=0x7fb14fd38700) at pthread_create.c:309
  365. #4  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  366.  
  367. Thread 20 (Thread 0x7fb14fdb0700 (LWP 1557)):
  368. #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  369. #1  0x00007fb195006f1e in switch_user_sql_thread (thread=0x7fb16c029494, obj=0x7fb16c0292d0)
  370.     at src/switch_core_sqldb.c:2135
  371. #2  0x00007fb195303000 in dummy_worker (opaque=0x7fb18c1616c0) at threadproc/unix/thread.c:151
  372. #3  0x00007fb19452c064 in start_thread (arg=0x7fb14fdb0700) at pthread_create.c:309
  373. #4  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  374.  
  375. Thread 19 (Thread 0x7fb14fd74700 (LWP 1556)):
  376. #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  377. #1  0x00007fb195006f1e in switch_user_sql_thread (thread=0x7fb1600299c4, obj=0x7fb1600297f0)
  378.     at src/switch_core_sqldb.c:2135
  379. #2  0x00007fb195303000 in dummy_worker (opaque=0x7fb14fffe6c0) at threadproc/unix/thread.c:151
  380. #3  0x00007fb19452c064 in start_thread (arg=0x7fb14fd74700) at pthread_create.c:309
  381. #4  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  382.  
  383. Thread 18 (Thread 0x7fb18c516700 (LWP 1555)):
  384. #0  0x00007fb193c04c03 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81
  385. #1  0x00007fb18c8ef37c in su_epoll_port_wait_events (self=0x7fb1480008c0, tout=<optimized out>)
  386.     at su_epoll_port.c:495
  387. #2  0x00007fb18c8ee916 in su_base_port_run (self=0x7fb1480008c0) at su_base_port.c:349
  388. #3  0x00007fb18c8f35e0 in su_pthread_port_clone_main (varg=0x7fb18c641790) at su_pthread_port.c:343
  389. #4  0x00007fb19452c064 in start_thread (arg=0x7fb18c516700) at pthread_create.c:309
  390. #5  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  391.  
  392. ---Type <return> to continue, or q <return> to quit---
  393. Thread 17 (Thread 0x7fb18c552700 (LWP 1554)):
  394. #0  0x00007fb193c04c03 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81
  395. #1  0x00007fb18c8ef37c in su_epoll_port_wait_events (self=0x7fb1540008c0, tout=<optimized out>)
  396.     at su_epoll_port.c:495
  397. #2  0x00007fb18c8ee916 in su_base_port_run (self=0x7fb1540008c0) at su_base_port.c:349
  398. #3  0x00007fb18c8f35e0 in su_pthread_port_clone_main (varg=0x7fb18c67d790) at su_pthread_port.c:343
  399. #4  0x00007fb19452c064 in start_thread (arg=0x7fb18c552700) at pthread_create.c:309
  400. #5  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  401.  
  402. Thread 16 (Thread 0x7fb18c58e700 (LWP 1553)):
  403. #0  0x00007fb193c04c03 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81
  404. #1  0x00007fb18c8ef37c in su_epoll_port_wait_events (self=0x7fb1500008c0, tout=<optimized out>)
  405.     at su_epoll_port.c:495
  406. #2  0x00007fb18c8ee916 in su_base_port_run (self=0x7fb1500008c0) at su_base_port.c:349
  407. #3  0x00007fb18c8f35e0 in su_pthread_port_clone_main (varg=0x7fb18c6b9790) at su_pthread_port.c:343
  408. #4  0x00007fb19452c064 in start_thread (arg=0x7fb18c58e700) at pthread_create.c:309
  409. #5  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  410.  
  411. Thread 15 (Thread 0x7fb18c5ca700 (LWP 1552)):
  412. #0  0x00007fb193c04c03 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81
  413. #1  0x00007fb18c8ef37c in su_epoll_port_wait_events (self=0x7fb15c0008c0, tout=<optimized out>)
  414.     at su_epoll_port.c:495
  415. #2  0x00007fb18c8ee916 in su_base_port_run (self=0x7fb15c0008c0) at su_base_port.c:349
  416. #3  0x00007fb18c8f35e0 in su_pthread_port_clone_main (varg=0x7fb18c6f5790) at su_pthread_port.c:343
  417. #4  0x00007fb19452c064 in start_thread (arg=0x7fb18c5ca700) at pthread_create.c:309
  418. #5  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  419.  
  420. Thread 14 (Thread 0x7fb18c606700 (LWP 1551)):
  421. #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  422. #1  0x00007fb1952fcfad in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>)
  423.     at locks/unix/thread_cond.c:68
  424. #2  0x00007fb1952f4ab1 in apr_queue_pop (queue=queue@entry=0x7fb18c758ad8, data=data@entry=0x7fb18c605e30)
  425.     at misc/apr_queue.c:276
  426. #3  0x00007fb194fe3af5 in switch_queue_pop (queue=queue@entry=0x7fb18c758ad8, data=data@entry=0x7fb18c605e30)
  427.     at src/switch_apr.c:1159
  428. #4  0x00007fb18c831cd0 in sofia_msg_thread_run (thread=<optimized out>, obj=0x7fb18c758ad8) at sofia.c:2282
  429. #5  0x00007fb195303000 in dummy_worker (opaque=0xaf1c90) at threadproc/unix/thread.c:151
  430. #6  0x00007fb19452c064 in start_thread (arg=0x7fb18c606700) at pthread_create.c:309
  431. #7  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  432.  
  433. Thread 13 (Thread 0x7fb18c642700 (LWP 1550)):
  434. #0  0x00007fb193c04c03 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81
  435. #1  0x00007fb18c8ef37c in su_epoll_port_wait_events (self=0x7fb1580008c0, tout=<optimized out>)
  436.     at su_epoll_port.c:495
  437. #2  0x00007fb18c8eea72 in su_base_port_step (self=0x7fb1580008c0, tout=1000) at su_base_port.c:467
  438. #3  0x00007fb18c81def3 in sofia_profile_thread_run (thread=0x19, obj=0xb07860) at sofia.c:3383
  439. #4  0x00007fb195303000 in dummy_worker (opaque=0xb0a130) at threadproc/unix/thread.c:151
  440. #5  0x00007fb19452c064 in start_thread (arg=0x7fb18c642700) at pthread_create.c:309
  441. #6  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  442.  
  443. Thread 12 (Thread 0x7fb18c67e700 (LWP 1549)):
  444. #0  0x00007fb193c04c03 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81
  445. #1  0x00007fb18c8ef37c in su_epoll_port_wait_events (self=0x7fb1600008c0, tout=<optimized out>)
  446. ---Type <return> to continue, or q <return> to quit---
  447.     at su_epoll_port.c:495
  448. #2  0x00007fb18c8eea72 in su_base_port_step (self=0x7fb1600008c0, tout=1000) at su_base_port.c:467
  449. #3  0x00007fb18c81def3 in sofia_profile_thread_run (thread=0x15, obj=0xb00c60) at sofia.c:3383
  450. #4  0x00007fb195303000 in dummy_worker (opaque=0xb037a8) at threadproc/unix/thread.c:151
  451. #5  0x00007fb19452c064 in start_thread (arg=0x7fb18c67e700) at pthread_create.c:309
  452. #6  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  453.  
  454. Thread 11 (Thread 0x7fb18c6ba700 (LWP 1548)):
  455. #0  0x00007fb193c04c03 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81
  456. #1  0x00007fb18c8ef37c in su_epoll_port_wait_events (self=0x7fb16c0008c0, tout=<optimized out>)
  457.     at su_epoll_port.c:495
  458. #2  0x00007fb18c8eea72 in su_base_port_step (self=0x7fb16c0008c0, tout=1000) at su_base_port.c:467
  459. #3  0x00007fb18c81def3 in sofia_profile_thread_run (thread=0x11, obj=0xafa5b0) at sofia.c:3383
  460. #4  0x00007fb195303000 in dummy_worker (opaque=0xafcc48) at threadproc/unix/thread.c:151
  461. #5  0x00007fb19452c064 in start_thread (arg=0x7fb18c6ba700) at pthread_create.c:309
  462. #6  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  463.  
  464. Thread 10 (Thread 0x7fb18c6f6700 (LWP 1547)):
  465. #0  0x00007fb193c04c03 in epoll_wait () at ../sysdeps/unix/syscall-template.S:81
  466. #1  0x00007fb18c8ef37c in su_epoll_port_wait_events (self=0x7fb1680008c0, tout=<optimized out>)
  467.     at su_epoll_port.c:495
  468. #2  0x00007fb18c8eea72 in su_base_port_step (self=0x7fb1680008c0, tout=1000) at su_base_port.c:467
  469. #3  0x00007fb18c81def3 in sofia_profile_thread_run (thread=0xd, obj=0xaf3240) at sofia.c:3383
  470. #4  0x00007fb195303000 in dummy_worker (opaque=0xaf65d0) at threadproc/unix/thread.c:151
  471. #5  0x00007fb19452c064 in start_thread (arg=0x7fb18c6f6700) at pthread_create.c:309
  472. #6  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  473.  
  474. Thread 9 (Thread 0x7fb18d898700 (LWP 1546)):
  475. #0  0x00007fb194532c8d in accept () at ../sysdeps/unix/syscall-template.S:81
  476. #1  0x00007fb195301b26 in apr_socket_accept (new=0x7fb18d897da0, sock=0xa84878,
  477.     connection_context=0x7fb170000aa8) at network_io/unix/sockets.c:191
  478. #2  0x00007fb1950e3692 in msrp_listener (thread=0xa84ad0, obj=0x7fb1955ef578 <globals+120>)
  479.     at src/switch_msrp.c:1379
  480. #3  0x00007fb195303000 in dummy_worker (opaque=0xa84ad0) at threadproc/unix/thread.c:151
  481. #4  0x00007fb19452c064 in start_thread (arg=0x7fb18d898700) at pthread_create.c:309
  482. #5  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  483.  
  484. Thread 8 (Thread 0x7fb18d85c700 (LWP 1545)):
  485. #0  0x00007fb194532c8d in accept () at ../sysdeps/unix/syscall-template.S:81
  486. #1  0x00007fb195301b26 in apr_socket_accept (new=0x7fb18d85bda0, sock=0xa84420,
  487.     connection_context=0x7fb17c0013c8) at network_io/unix/sockets.c:191
  488. #2  0x00007fb1950e3692 in msrp_listener (thread=0xa84778, obj=0x7fb1955ef558 <globals+88>)
  489.     at src/switch_msrp.c:1379
  490. #3  0x00007fb195303000 in dummy_worker (opaque=0xa84778) at threadproc/unix/thread.c:151
  491. #4  0x00007fb19452c064 in start_thread (arg=0x7fb18d85c700) at pthread_create.c:309
  492. #5  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  493.  
  494. Thread 7 (Thread 0x7fb18d8d4700 (LWP 1543)):
  495. #0  pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238
  496. #1  0x00007fb1952fd012 in apr_thread_cond_timedwait (cond=0xa7e448, mutex=0xa7e3f8, timeout=timeout@entry=500000)
  497.     at locks/unix/thread_cond.c:89
  498. #2  0x00007fb1952f4b9c in apr_queue_pop_timeout (queue=0xa7e3b8, data=data@entry=0x7fb18d8d3e40,
  499.     timeout=timeout@entry=500000) at misc/apr_queue.c:339
  500. ---Type <return> to continue, or q <return> to quit---
  501. #3  0x00007fb194fe3b05 in switch_queue_pop_timeout (queue=<optimized out>, data=data@entry=0x7fb18d8d3e40,
  502.     timeout=timeout@entry=500000) at src/switch_apr.c:1164
  503. #4  0x00007fb19505e68f in switch_scheduler_task_thread (thread=<optimized out>, obj=<optimized out>)
  504.     at src/switch_scheduler.c:188
  505. #5  0x00007fb195303000 in dummy_worker (opaque=0x7fb18dabd5d8) at threadproc/unix/thread.c:151
  506. #6  0x00007fb19452c064 in start_thread (arg=0x7fb18d8d4700) at pthread_create.c:309
  507. #7  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  508.  
  509. Thread 6 (Thread 0x7fb18dafa700 (LWP 1542)):
  510. #0  0x00007fb193bfd893 in select () at ../sysdeps/unix/syscall-template.S:81
  511. #1  0x00007fb195304565 in apr_sleep (t=<optimized out>) at time/unix/time.c:246
  512. #2  0x00007fb1950d38fc in do_sleep (t=<optimized out>) at src/switch_time.c:164
  513. #3  0x00007fb19500cb9d in switch_core_sql_db_thread (thread=<optimized out>, obj=<optimized out>)
  514.     at src/switch_core_sqldb.c:1395
  515. #4  0x00007fb195303000 in dummy_worker (opaque=0x7fb18df4a748) at threadproc/unix/thread.c:151
  516. #5  0x00007fb19452c064 in start_thread (arg=0x7fb18dafa700) at pthread_create.c:309
  517. #6  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  518.  
  519. Thread 5 (Thread 0x7fb18db36700 (LWP 1541)):
  520. #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  521. #1  0x00007fb195006f1e in switch_user_sql_thread (thread=0xa7445c, obj=0xa742b0) at src/switch_core_sqldb.c:2135
  522. #2  0x00007fb195303000 in dummy_worker (opaque=0x7fb18dbfa6c0) at threadproc/unix/thread.c:151
  523. #3  0x00007fb19452c064 in start_thread (arg=0x7fb18db36700) at pthread_create.c:309
  524. #4  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  525.  
  526. Thread 4 (Thread 0x7fb18de86700 (LWP 1540)):
  527. #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  528. #1  0x00007fb1952fcfad in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>)
  529.     at locks/unix/thread_cond.c:68
  530. #2  0x00007fb1952f4ab1 in apr_queue_pop (queue=0x7fb18e0a9548, data=data@entry=0x7fb18de85e40)
  531.     at misc/apr_queue.c:276
  532. #3  0x00007fb194fe3af5 in switch_queue_pop (queue=<optimized out>, data=data@entry=0x7fb18de85e40)
  533.     at src/switch_apr.c:1159
  534. #4  0x00007fb1950c9cd0 in log_thread (t=<optimized out>, obj=<optimized out>) at src/switch_log.c:294
  535. #5  0x00007fb195303000 in dummy_worker (opaque=0x7fb18df4a5a8) at threadproc/unix/thread.c:151
  536. #6  0x00007fb19452c064 in start_thread (arg=0x7fb18de86700) at pthread_create.c:309
  537. #7  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  538.  
  539. Thread 3 (Thread 0x7fb18df87700 (LWP 1539)):
  540. #0  0x00007fb193bfbaed in poll () at ../sysdeps/unix/syscall-template.S:81
  541. #1  0x00007fb19506cbea in switch_wait_socklist (waitlist=0x7fb18df86bc0, len=2, ms=100)
  542.     at src/switch_utils.c:2983
  543. #2  0x00007fb14f4ec6e7 in profile_one_loop (profile=0x7fb14f4bf038) at mod_verto.c:4359
  544. #3  0x00007fb14f4ed210 in runtime (profile=<optimized out>) at mod_verto.c:4487
  545. #4  profile_thread (thread=0x7fb1840008c0, obj=0x7fb14f4bf038) at mod_verto.c:5126
  546. #5  0x00007fb19500e1ad in switch_core_session_thread_pool_worker (thread=0x9f3b00, obj=0x2)
  547.     at src/switch_core_session.c:1758
  548. #6  0x00007fb195303000 in dummy_worker (opaque=0x9f3b00) at threadproc/unix/thread.c:151
  549. #7  0x00007fb19452c064 in start_thread (arg=0x7fb18df87700) at pthread_create.c:309
  550. #8  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  551.  
  552. Thread 2 (Thread 0x7fb18e05a700 (LWP 1538)):
  553. #0  pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:185
  554. ---Type <return> to continue, or q <return> to quit---
  555. #1  0x00007fb1952fcfad in apr_thread_cond_wait (cond=<optimized out>, mutex=<optimized out>)
  556.     at locks/unix/thread_cond.c:68
  557. #2  0x00007fb1952f4ab1 in apr_queue_pop (queue=0x835cd8, data=0x7fb18e059e20) at misc/apr_queue.c:276
  558. #3  0x00007fb1950707a6 in switch_event_dispatch_thread (thread=0x7fb18e059e20, obj=0x835cd8)
  559.     at src/switch_event.c:324
  560. #4  0x00007fb195303000 in dummy_worker (opaque=0x7fb18e0a92c0) at threadproc/unix/thread.c:151
  561. #5  0x00007fb19452c064 in start_thread (arg=0x7fb18e05a700) at pthread_create.c:309
  562. #6  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  563.  
  564. Thread 1 (Thread 0x7fb195c89700 (LWP 1537)):
  565. #0  0x00007fb193bfd893 in select () at ../sysdeps/unix/syscall-template.S:81
  566. #1  0x00007fb195304565 in apr_sleep (t=<optimized out>) at time/unix/time.c:246
  567. #2  0x00007fb1950d38fc in do_sleep (t=<optimized out>) at src/switch_time.c:164
  568. #3  0x00007fb194fff4c5 in pool_thread (thread=<optimized out>, obj=<optimized out>)
  569.     at src/switch_core_memory.c:569
  570. #4  0x00007fb195303000 in dummy_worker (opaque=0x7fb195cebc40) at threadproc/unix/thread.c:151
  571. #5  0x00007fb19452c064 in start_thread (arg=0x7fb195c89700) at pthread_create.c:309
  572. #6  0x00007fb193c0462d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
  573. (gdb)
  574. (gdb)