Commit Graph

6652 Commits

Author SHA1 Message Date
George Joseph
f55e20ae83 Merge "http.c: Reload TLS even if http.conf hasn't changed" 2018-10-04 07:23:49 -05:00
George Joseph
967fffb569 Merge "core/frame: Fix ast_frdup() and ast_frisolate() for empty text frames" 2018-10-03 13:55:24 -05:00
Corey Farrell
932d0a40cf astobj2: Comment on OBJ_NOLOCK in ao2_container_clone.
The test for OBJ_NOLOCK looks wrong but it isn't.  Add comments to
prevent confusion.

Change-Id: I9662b82eb39e7627a1f1944fd18f967a2b987344
2018-10-03 11:03:48 -05:00
George Joseph
bc9a7c419c Merge "astobj2: Record lock usage to refs log when DEBUG_THREADS is enabled." 2018-10-03 10:17:27 -05:00
George Joseph
24dc6ee8ae Merge "loader: Fix result of module reload error." 2018-10-03 10:17:07 -05:00
George Joseph
910f5dbd44 Merge "core: Disable astobj2 locking for some common objects." 2018-10-03 10:16:20 -05:00
Sean Bright
286339aa34 http.c: Reload TLS even if http.conf hasn't changed
There is currently no way to indicate to Asterisk that TLS certificates
and/or keys have been updated other than by modifying http.conf or
restarting Asterisk.

There is already code in main/tcptls.c that determines if a reload is
actually necessary based on the hashes of the certicate and dependent
files, so this change merely gives us a way to request a reload without
explicitly modifying http.conf.

Change-Id: Ie795420dcc7eb3d91336820688a29adbcc321276
2018-10-03 08:35:12 -05:00
Corey Farrell
cacbe32534 core: Disable astobj2 locking for some common objects.
* ACO options
* Indications
* Module loader ref_debug object
* Media index info and variants
* xmldoc items

These allocation locations were identified using reflocks.py on the
master branch.

Change-Id: Ie999b9941760be3d1946cdb6e30cb85fd97504d8
2018-10-02 16:47:23 -04:00
Corey Farrell
b25a261aa5 loader: Fix result of module reload error.
When a module reload fails we never set AST_MODULE_RELOAD_ERROR.  This
caused reload failures to incorrectly report 'No module found'.

Change-Id: I5f3953e0f7d135e53ec797f24c97ee3f73f232e7
2018-10-02 14:12:58 -04:00
Corey Farrell
e4cf513f81 loader: Improve error handling.
* Display list of unavailable dependencies when they cause another
  module to fail loading.
* When a module declines to load find all modules which depend on it so
  they can be declined and listed together.
* Prevent retry of declined modules during startup.
* When a module fails to dlopen try loading it with RTLD_LAZY so we can
  attempt to display the list of missing dependencies.

These changes are meant to reduce logger spam that is caused when a
module has many dependencies and declines to load.  This also fixes some
error paths which failed to recognize required modules.

Module load/start errors are delayed until the end of loader startup.

Change-Id: I046052c71331c556c09d39f47a3b92975f3e1758
2018-10-02 13:18:12 -04:00
neutrino88
24cece660b core/frame: Fix ast_frdup() and ast_frisolate() for empty text frames
If a channel creates an AST_TEXT_FRAME with datalen == 0, the ast_frdup()
and ast_frisolate() functions could create a clone frame with an invalid
data.ptr which would cause a crash.  The proposed fix is to make sure that
for such empty text frames, ast_frdup() and ast_frisolate() return cloned
text frames with a valid data.ptr.

ASTERISK-28076
Reported by: Emmanuel BUU
Tested by: Emmanuel BUU

Change-Id: Ib882dd028598f13c4c233edbfdd7e54ad44a68e9
2018-10-02 11:25:25 -05:00
Corey Farrell
13df745278 astobj2: Record lock usage to refs log when DEBUG_THREADS is enabled.
When DEBUG_THREADS is enabled we can know if the astobj2 mutex / rwlock
was ever used, so it can be recorded in the REF_DEBUG destructor entry.

Create contrib/scripts/reflocks.py to process locking used by
allocator.  This can be used to identify places where
AO2_ALLOC_OPT_LOCK_NOLOCK should be used to reduce memory usage.

Change-Id: I2e3cd23336a97df2692b545f548fd79b14b53bf4
2018-10-01 22:27:30 -04:00
Jenkins2
252e4a8fd4 Merge "astobj2: Reduce memory overhead." 2018-10-01 09:02:37 -05:00
Joshua Colp
69f4740fa1 Merge "lock: Improve performance of DEBUG_THREADS." 2018-10-01 08:32:49 -05:00
Corey Farrell
205c6be895 lock: Improve performance of DEBUG_THREADS.
Add a volatile flag to lock tracking structures so we only need to use
the global lock when first initializing tracking.

Additionally add support for DEBUG_THREADS_LOOSE_ABI.  This is used by
astobj2.c to eliminate storage for tracking fields when DEBUG_THREADS is
not defined.

Change-Id: Iabd650908901843e9fff47ef1c539f0e1b8cb13b
2018-09-28 14:42:04 -04:00
George Joseph
930cbec67d Merge "config.c: Cleanup AST_INCLUDE_GLOB" 2018-09-28 13:16:17 -05:00
Corey Farrell
62a0db2df1 astobj2: Reduce memory overhead.
Reduce options to 2-bit field, magic to 30 bit field.  Move ref_counter
next to options and explicitly use int32_t so the fields will pack.

This reduces memory overhead for every ao2 object by 8 bytes on x86_64.

Change-Id: Idc1baabb35ec3b3d8de463c4fa3011eaf7fcafb5
2018-09-27 18:02:04 -04:00
Sean Bright
ac23e5ad48 config.c: Cleanup AST_INCLUDE_GLOB
* In main/config.c, AST_INCLUDE_GLOB is fixed to '1' making the #ifdefs
  pointless.

* In utils/extconf.c, AST_INCLUDE_GLOB is never defined so there is a
  lot of dead code.

Change-Id: I1bad1a46d7466ddf90d52cc724e997195495226c
2018-09-27 15:07:08 -05:00
Corey Farrell
39bf9881e0 astobj2: Fix shutdown order.
When REF_DEBUG and AO2_DEBUG are both enabled we closed the refs log
before we shutdown astobj2_container.  This caused the AO2_DEBUG
container registration container to be reported as a leak.

Change-Id: If9111c4c21c68064b22c546d5d7a41fac430430e
2018-09-27 06:33:22 -04:00
George Joseph
aa10d6d5eb Merge "jansson: Backport fixes to bundled, use json_vsprintf if available." 2018-09-26 11:09:50 -05:00
George Joseph
de14ed0b93 Merge "rtp_engine: rtcp_report_to_json can overflow the ssrc integer value" 2018-09-26 08:02:28 -05:00
Corey Farrell
adf539b2f0 jansson: Backport fixes to bundled, use json_vsprintf if available.
Use json_vsprintf from versions which contain fix for va_copy leak.

Apply fixes from jansson master:
* va_copy leak fix.
* Avoid potential invalid memory read in json_pack.
* Rename variable that shadowed another.

Change-Id: I7522e462d2a52f53010ffa1e7d705c666ec35539
2018-09-24 18:00:26 -05:00
Corey Farrell
93777faf36 json: Take advantage of new API's.
* Use "o*" format specifier for optional fields in ast_json_party_id.
* Stop using ast_json_deep_copy on immutable objects, it is now thread
  safe to just use ast_json_ref.

Additional changes to ast_json_pack calls in the vicinity:
* Use "O" when an object needs to be bumped.  This was previously
  avoided as it was not thread safe.
* Use "o?" and "O?" to replace NULL with ast_json_null().  The
  "?" is a new feature of ast_json_pack starting with Asterisk 16.

Change-Id: I8382d28d7d83ee0ce13334e51ae45dbc0bdaef48
2018-09-24 15:47:37 -04:00
George Joseph
06c0676da0 app_voicemail: Cleanup mailbox topic and cache
app_voicemail wasn't properly cleaning up the stasis cache or the
mwi topic pool when the module was unloaded or when a user was
deleted as a result of a reload.  This resulted in leaks in both
areas.

* app_voicemail now calls ast_delete_mwi_state_full when it frees
  a user structure and ast_delete_mwi_state_full in turn now calls
  the new stasis_topic_pool_delete_topic function to clear the topic
  from the pool.

Change-Id: Ide23144a4a810e7e0faad5a8e988d15947965df8
2018-09-24 10:10:48 -05:00
George Joseph
9ec08d7532 Merge "stasis: Add function to delete topic from pool" 2018-09-24 09:29:14 -05:00
Kevin Harwell
31fba4e869 rtp_engine: rtcp_report_to_json can overflow the ssrc integer value
When writing an RTCP report to json the code attempts to pack the "ssrc" and
"source_ssrc" unsigned integer values as a signed int value type. This of course
means if the ssrc's unsigned value is greater than that which can fit into a
signed integer value it gets converted to a negative number. Subsequently, the
negative value goes out in the json report.

This patch now packs the value as a json_int_t, which is the widest integer type
available on a given system. This should make it so the value no longer
overflows.

Note, this was caught by two failing tests hep/rtcp-receiver/ and
hep/rtcp-sender.

Change-Id: I2af275286ee5e795b79f0c3d450d9e4b28e958b0
2018-09-21 18:06:21 -04:00
George Joseph
4d51a8e05b channel.c: Address stack overflow in does_id_conflict()
does_id_conflict() was passing a pointer to an int to a callback
that expected a pointer to a size_t.

Found by the Address Sanitizer.

Change-Id: I0ff542067eef63a14a60301654d65d34fe2ad503
2018-09-21 15:32:31 -05:00
George Joseph
d277db4a38 stasis: Add function to delete topic from pool
There's been a long standing leak when using topic pools.  The
topics in the pool get cleaned up when the last pool reference is
released but you can't remove a topic specifically.  If you reloaded
app_voicemail for instance, and mailboxes went away, their topics
were left in the pool.

* Added stasis_topic_pool_delete_topic() so modules can clean up
  topics from pools.
* Registered the topic pool containers so it can be examined from
  the CLI when AO2_DEBUG is enabled.  They'll be named
  "<topic_pool_name>-pool".

Change-Id: Ib7957951ee5c9b9b4482af7b9b4349112d62bc25
2018-09-20 12:13:22 -06:00
George Joseph
e3e24a0844 Merge "stasis: No need to keep a stasis type ref in a stasis msg or cache object." 2018-09-20 13:08:45 -05:00
Joshua Colp
b793f8ccf2 Merge "stasis_cache: Stop caching stasis subscription change messages" 2018-09-20 09:43:26 -05:00
Richard Mudgett
c99a9b228b stasis: No need to keep a stasis type ref in a stasis msg or cache object.
Stasis message types are global ao2 objects and we make stasis messages
and cache entries hold references to them.  Since there are currently
situations where cache objects are never deleted, the reference count on
the types can exceed 100000 and generate a FRACK assertion message.  The
stasis message cache could conceivably also have that many messages
legitimately on large systems.

The only down side to not holding the message type ref in the stasis
message is it only makes a crash either at shutdown or when manually
unloading a busy module slightly more likely.  However, this is more
exposing a pre-existing stasis shutdown ordering issue than a problem with
not holding a message type ref in stasis messages.

* Made stasis messages and cache entries no longer hold a ref to the
message type.

Change-Id: Ibaa28efa8d8ad3836f0c65957192424c7f561707
2018-09-19 12:32:46 -05:00
Richard Mudgett
79e3becc5d stasis_message.c: Don't create immutable stasis objects with locks.
* Create the stasis message object without a lock as it is immutable.
* Create the stasis message type object without a lock as it is immutable.
* Creating the stasis message type could crash if the passed in type name
is NULL and REF_DEBUG is enabled.  Added missing NULL check when passing
the ao2 object tag string.

Change-Id: I28763c58bb9f0b427c11971d0103bf94055e7b32
2018-09-18 13:18:15 -05:00
George Joseph
5ec6d2c33e stasis_cache: Stop caching stasis subscription change messages
Since app_voicemail no longer uses the cache to maintain its state
there is no longer a need to cache these messages.

ASTERISK-27121

Change-Id: I321c708505f5ad8d00e1b0afc4c27dc2ac12ecb4
2018-09-14 06:04:58 -05:00
Joshua Colp
2969d2ac5c Merge "manager: Set AMI event "Newexten" to the EVENT_FLAG_DIALPLAN class" 2018-09-12 11:01:25 -05:00
lvl
012272a114 manager: Set AMI event "Newexten" to the EVENT_FLAG_DIALPLAN class
The documentation already specified EVENT_FLAG_DIALPLAN for this
event, but the implementation was using EVENT_FLAG_CALL.

Using EVENT_FLAG_DIALPLAN allows AMI clients to opt out of receiving
this highly verbose event.

ASTERISK-28033

Change-Id: I45b3119f30e4dbc17b49831f2b1a4f2c1beadafe
2018-09-12 11:22:02 -03:00
Jenkins2
29d4109b3c Merge "core: Don't stop generators when writing RTCP frames." 2018-09-07 07:02:38 -05:00
Joshua Colp
c1942e1cca Merge "stasis_cache: Prune stasis_subscription_change messages" 2018-09-07 05:40:36 -05:00
Joshua Colp
f97d92bd0a core: Don't stop generators when writing RTCP frames.
Generators provide such functionality as tone generation or
silence generation. RTCP frames provide RTCP information and
should not stop generators from operating.

ASTERISK-28005

Change-Id: Ieadada07b068a7aa426e8763f1b73a18e1ac34a9
2018-09-06 17:08:55 -05:00
George Joseph
16df182155 Merge "http.c: Give HTTP error response when received lines are too long." 2018-09-06 11:49:25 -05:00
George Joseph
0dd8ab3532 stasis_cache: Prune stasis_subscription_change messages
The stasis cache provides a way to reconstruct the current state
of topic subscribers.  Unfortunately, since every subscribe and
unsubscribe is cached, the cache continues to grow unabated while
asterisk is running.  This patch removes subscribe messages from
the cache when the corresponding unsubscribe is received.

This patch also registers the cache containers with ao2 so that if
AO2_DEBUG is turned on, you can list the container and get its
stats from the CLI.

ASTERISK-27121

Change-Id: I3d18905e477f3721815da91f30da8d3fbb2d4f56
2018-09-05 13:52:17 -05:00
Richard Mudgett
e387750104 http.c: Give HTTP error response when received lines are too long.
Added a check when we receive a HTTP request line or header line that is
too long.  We now return an error response to the sender because we are
not able to process the request.

Change-Id: I6df2705435fd7dde4d5d3bdf7acec859cfb7c12d
2018-08-30 17:22:38 -05:00
Richard Mudgett
f657793ee4 iostream.c: Fix ast_iostream_gets() needlessly returning failure.
Providing a buffer larger than the internal buffer of ast_iostream_gets()
fails to get lines longer than the internal buffer.

* Made ast_iostream_gets() fill the supplied buffer with read data until
either a '\n' is found or the supplied buffer is filled just like fgets().

Change-Id: If18b3f6ee500e22f0633a68779ed09f7e0f305ed
2018-08-30 17:12:11 -05:00
Jaco Kroon
a74f8e51a6 AMI: be less verbose when adding HTTP headers to AMI/HTTP messages.
All HTTP/AMI message headers are being sent to the verbose channel.
There are multiple places this is happening.  Consolidate the loop into
a function.  Drop the debug/verbose message.

Convert to using ast_asprintf to perform the length calculation, memory
allocation and snprintf all in one step.

Change-Id: Ic45e673fde05bd544be95ad5cdbc69518207c1a1
2018-08-23 21:43:38 +02:00
Richard Mudgett
5ec27d5206 AMI: Remove docs for nonexistent AMI ContactStatus event headers
Change-Id: I5736965c64c44338f7330e85a24bb46818607f19
2018-08-20 12:32:58 -05:00
Jenkins2
5241a53acd Merge "Build System: Improve ccache matching for different menuselect options." 2018-08-14 13:41:32 -05:00
Joshua Colp
455ca1095e stasis: Reduce calculation of stasis message type hash.
When the stasis cache is used a hash is calculated for
retrieving or inserting messages. This change calculates
a hash when the message type is initialized that is then
used each time needed. This ensures that the hash is
calculated only once for the message type.

Change-Id: I4fe6bfdafb55bf5c322dd313fbd8c32cce73ef37
2018-08-06 11:18:04 -05:00
Corey Farrell
a10a3aff6a Build System: Improve ccache matching for different menuselect options.
Changing any Menuselect option in the `Compiler Flags` section causes a
full rebuild of the Asterisk source tree.  Every enabled option causes
a #define to be added to buildopts.h, thus breaking ccache caching for
every source file that includes "asterisk.h".  In most cases each option
only applies to one or two files.  Now we only define those options for
the specific sources which use them, this causes much better cache
matching when working with multiple builds.  For example testing code
with an without MALLOC_DEBUG will now use just over half the ccache
size, only main/astmm.o will have two builds cached instead of every
file.

Reorder main/Makefile so _ASTCFLAGS set on specific object files are all
together, sorted by filename.  Stop adding -DMALLOC_DEBUG to CFLAGS of
bundled pjproject, this define is no longer used by any header so only
serves to break cache.

The only code change is a slight adjustment to how main/astmm.c is
initialized.  Initialization functions always exist so main/asterisk.c
can call them unconditionally.  Additionally rename the astmm
initialization functions so they are not exported.

Change-Id: Ie2085237a964f6e1e6fff55ed046e2afff83c027
2018-08-01 12:01:15 -04:00
Joshua Colp
4febeaefa6 Merge "devicestate: Don't create topic when change isn't cached." 2018-07-27 06:09:50 -05:00
Corey Farrell
709f4b81e7 loader: Process dependencies for built-in modules.
With the new module loader it was missed that built-in modules never
parsed dependencies from mod->info into vectors of mod.  This caused
manager to be initialized before acl (named_acl).  If manager.conf
used any named ACL's they would not be found and result in no ACL being
applied to the AMI user.

In addition to the manager ACL fix this adds "extconfig" to all builtin
modules which support realtime configuration.  This only matters if one
of the builtin modules is configured with 'preload', depending on
"extconfig" will cause config.c to automatically be initialize during
the preload stage.

Change-Id: I482ed6bca6c1064b05bb538d7861cd7a4f02d9fc
2018-07-26 14:29:18 -05:00
Kevin Harwell
783bff0637 json.c: improve ast_json_to_ast_variables performance
When converting from a json object to an ast variables list the conversion
algorithm was doing a complete traversal of the entire variables list for
every item appended from the json structure.

This patch makes it so the list is no longer traversed for each new ast
variable being appended.

Change-Id: I8bf496a1fc449485150d6db36bfc0354934a3977
2018-07-25 15:37:28 -05:00