Commit Graph

13 Commits

Author SHA1 Message Date
Kevin Harwell
26713dc88b ast_coredumper: add Asterisk information dump
This patch makes it so ast_coredumper now outputs the following information to
a *-info.txt file when processing a core file:

  asterisk version and "built by" string
  BUILD_OPTS
  system start, and last reloaded date/time
  taskprocessor list
  equivalent of "bridge show all"
  equivalent of "core show channels verbose"

Also a slight modification was made when trying to obtain the pid(s) of a
running Asterisk. If it fails to retrieve any it now reports an error.

Change-Id: I54f35c19ab69b8f8dc78cc933c3fb7c99cef346b
2020-03-26 08:27:55 -05:00
George Joseph
809e836265 ast_coredumper: Refactor the pid determination process
In order to get a dump of the running process, we need to find the
pid of the main asterisk process.  This can be tricky if there are
also instances of "asterisk -r" running or if an alternate location
for asterisk.conf was specified on the command line with the -C
option that also specified an alternation location for the pid file.

So now...

1. We find the asterisk executable with "which" or the --asterisk-bin
   command line option.
2. If there's only 1 process with an executable path that matches,
   we use that pid.  If not...
3. We try "<asterisk-bin> -rx 'core show settings'" and parse the
   output to find the pidfile, then read that for the pid.  If that
   didn't work...
4. We get a list of all the pids matching <asterisk-bin> and look
   in /proc/<pid>/cmdline for a -C argument and retry the "core show
   settings" using the same -C option.  We can't parse the output
   of "ps" to get the -C path because it may contain spaces.  The
   contents of /proc/<pid>/cmdline are delimited by NULLs.  For BSDs
   we may have to mount /proc first. :(

ASTERISK-28221
Reported by: Andrew Nagy

Change-Id: I8aa1f3f912f949df2b5348908803c636bde1d57c
2018-12-24 14:17:38 -05:00
George Joseph
772d991a43 Merge "ast_coredumper: Remove .gdbinit file on exit" 2018-10-04 09:43:09 -05:00
Sean Bright
b2ed667712 ast_coredumper: Remove .gdbinit file on exit
Change-Id: I1297de78628773ca368e687c6f148bf74857cae9
2018-10-03 17:03:09 -05:00
George Joseph
a29cefe5b2 ast_coredumper: Don't use "declare -n"
Change-Id: I7ddfed4cd6549a0cd458e4d5cf9ac95d784de6cb
2018-10-03 15:30:22 -05:00
George Joseph
2bf26ce5ac ast_coredumper: Fix output directory and variable precedence
The OUTPUTDIR variable in ast_debug_tools.conf.sample is now set
to "/tmp" instead of "/some/directory".

Variables set on the command line or that are already in the
environment now take predecence over variables set in the config files.

ASTERISK-27846
Reported by: Ted G

Change-Id: Ie8baec52d531886bf5849ec1d59bb59dc87ad387
2018-05-24 12:59:37 -06:00
George Joseph
7e2128c8e6 ast_coredumper: Minor fixes
* Fix --tarball-config so the option doesn't cause an error.

* Allow for missing /etc/os-release.

* Add a sleep between tarballing the coredump and removing the
  output directory to allow the filesystem to settle.

Change-Id: I73e03b13087978bcc7f6bc9f45753990f82d9d77
2018-02-23 15:01:18 -06:00
George Joseph
cf1cb3345e ast_coredumper: Add ability to use directory other than /tmp
The OUTPUTDIR environment variable can now be set either in the
environment itself or in ast_debug_tools.conf.  If set, it's used
for all work products instead of /tmp.

Also added the --tarball-config option that includes the contents
of /etc/asterisk when either --tarball-coredumps or --tarball-results
are used.

Change-Id: I66b2553319df61caea5b313d084f51978f730b4c
2017-11-15 10:43:44 -05:00
Joshua Colp
d05391ed55 Merge "ast_coredumper: allow setting asterisk binary explicitly" 2017-10-31 06:48:16 -05:00
Tzafrir Cohen
022de525be ast_coredumper: allow setting asterisk binary explicitly
Adds an extra option, --asterisk-bin=<path> to ast_coredumper. If
provided, the binary given to gdb will be the parameter, rather than
asterisk from the PATH.

ASTERISK-27380 #close

Change-Id: I25f5b91eb75059b0fb2f142e468c26b283b0a9f3
2017-10-30 16:20:18 +02:00
George Joseph
4aec70690d ast_coredumper: Add gzipping of binaries and display of signal info
The --tarball-coredump option now creates a gzipped tarball of
coredumps processed, their results txt files and copies of
/etc/os-release, /usr/sbin/asterisk, /usr/lib(64)/libasterisk* and
/usr/lib(64)/asterisk as those files are needed to properly examine
the coredump.  The file will be named
/tmp/asterisk.<timestamp>.coredumps.tar.gz or
/tmp/asterisk-<uniqueid>.coredumps.tar.gz if --tarball-uniqueid was
specified.

Added dumps of *_siginfo to the top of the txt files so you can
tell what signal was invoked.

Change-Id: Ib9ee6d83592d4b1bc90cb3419a05376a88d1ded9
2017-10-25 12:28:21 -05:00
George Joseph
d16b3a9917 debug_utilities: Create ast_loggrabber
ast_loggrabber gathers log files from customizable search patterns,
optionally converts POSIX timestamps to a readable format and
tarballs the results.

Also a few tweaks were made to ast_coredumper.

Change-Id: I8bfe1468ada24c1344ce4abab7b002a59a659495
(cherry picked from commit c709152878)
2017-01-20 11:20:22 -06:00
George Joseph
0d53c91fba debug_utilities: Create the ast_coredumper utility
This utility allows easy manipulation of asterisk coredumps.

* Configurable search paths and patterns for existing coredumps
* Can generate a consistent coredump from the running instance
* Can dump the lock_infos table from a coredump
* Dumps backtraces to separate files...
  - thread apply 1 bt full -> <coredump>.thread1.txt
  - thread apply all bt -> <coredump>.brief.txt
  - thread apply all bt full -> <coredump>.full.txt
  - lock_infos table -> <coredump>.locks.txt
* Can tarball corefiles and optionally delete them after processing
* Can tarball results files and optionally delete them after processing
* Converts ':' in coredump and results file names '-' to facilitate
  uploading.  Jira for instance, won't accept file names with colons
  in them.

Tested on Fedora24+, Ubuntu14+, Debian6+, CentOS6+ and FreeBSD9+[1].

[1] For *BSDs, the "devel/gdb" package might have to be installed to
get a recent gdb.  The utility will check all instances of gdb
it finds in $PATH and if one isn't found that can run python, it
prints a friendly error.

Change-Id: I935d37ab9db85ef923f32b05579897f0893d33cd
(cherry picked from commit cb47b45560)
2017-01-11 12:11:45 -06:00