The threadpool_auto_increment test fails infrequently for a couple of
reasons
* The threadpool listener was notified of fewer tasks being pushed than
were actually pushed
* The "was_empty" flag was set to an unexpected value.
The problem is that the test pushes three tasks into the threadpool.
Test expects the threadpool to essentially gather those three tasks, and
then distribute those to the threadpool threads. It also expects that as
the tasks are pushed in, the threadpool listener is alerted immediately
that the tasks have been pushed. In reality, a task can be distributed
to the threadpool threads quicker than expected, meaning that the
threadpool has already emptied by the time each subsequent task is
pushed. In addition, the internal threadpool queue can be delayed so
that the threadpool listener is not alerted that a task has been pushed
even after the task has been executed.
From the test's point of view, there's no way to be able to predict
exactly the order that task execution/listener notifications will occur,
and there is no way to know which listener notifications will indicate
that the threadpool was previously empty.
For this reason, the test has been updated to only check the things it
can check. It ensures that all tasks get executed, that the threads go
idle after the tasks are executed, and that the listener is told the
proper number of tasks that were pushed.
Change-Id: I7673120d74adad64ae6894594a606e102d9a1f2c
When terminating the threads thrashing a sorcery memory cache each
would be told to stop and then we would wait on them. During at
least one thrashing test this was problematic due to the specific
usage pattern in use. It would take some time for termination of the
thread to occur.
This would occur due to contention between the threads retrieving
and the threads updating the cache. As the retrieving threads are
given priority it may be some time before the updating threads
are able to proceed.
This change makes it so all threads are told to stop and then each
are joined to ensure they stop. This way all the threads should
stop at around the same time instead of waiting for one to stop,
the next to stop, then the next, and so on. As a result of this
the execution time for each thrash test is much closer to their
expected value than previously seen as well.
Change-Id: I04a53470b0ea4170b8819180b0bd7475f3642827
test_dlinklists doesn't need to NOTICE everyone that every macro worked.
res_phoneprov doesn't need to VERBOSE everyone that a phoneprov extension or
provider was registered.
res_odbc was missing a newline at the end of one message.
Change-Id: I6c06361518ef3711821795e535acd439782a995e
This change makes the thread_timeout_thrash unit test wait for
each task to complete. This fixes the problem where the test would
prematurely end when all threads were gone and a new one had to be
started to handle the last task. It also increases the thrasing as
it is now more likely for each task to encounter the above scenario.
This also fixes a memory leak where the data for each task was not
being freed.
ASTERISK-25611 #close
Change-Id: I5017d621a4dc911f509074c16229b86bff2fb3c6
res_sorcery_realtime's search-by-regex callback performed a check to
ensure that the passed-in regex began with a caret (^). If it did not,
then no results would be returned.
This callback only started to become used when "like" support was added
to PJSIP CLI commands. The CLI command for listing objects would pass an
empty regex ("") to the sorcery backend if no "like" statement was
present. For most sorcery backends, this resulted in returning all
objects. However, for realtime, this resulted in returning no objects.
This commit seeks to fix the regression by removing the requirement from
res_sorcery_realtime for the passed-in-regex to begin with a caret.
ASTERISK-25689 #close
Reported by Marcelo Terres
Change-Id: I22b4dc5d7f3f11bb29ac2e42ef94682e9bab3b20
The cache_clear test was written to expect duplicate Stasis messages
sent from the technology endpoint to the all caching topic. This patch
fixes the test to no longer expect these duplicate messages.
ASTERISK-25137
Change-Id: I58075d70d6cdf42e792e0fb63ba624720bfce981
When appending all formats of a type all the codecs are iterated
and added. This operation was incorrectly adding the ast_format_none
format which is special in that it is supposed to be used when no
format is present. It shouldn't be appended.
ASTERISK-25535
Change-Id: I7b00f3bdf4a5f3022e483d6ece602b1e8b12827c
This change adds handling of dead worker threads when moving them
to be active. When this happens the worker thread is removed from
both the active and idle threads container. If no threads are able
to be moved to active then the pool grows as configured.
A unit test has also been added which thrashes the idle timeout
and thread activation to exploit any race conditions between the
two.
ASTERISK-25546 #close
Change-Id: I6c455f9a40de60d9e86458d447b548fb52ba1143
Some codecs that may be a third party library to Asterisk need to have
knowledge of the format attributes that were negotiated. Unfortunately,
when the great format migration of Asterisk 13 occurred, that ability
was lost.
This patch adds an API call, ast_format_attribute_get, to the core
format API, along with updates to the unit test to check the new API
call. A new callback is also now available for format attribute modules,
such that they can provide the format attribute values they manage.
Note that the API returns a void *. This is done as the format attribute
modules themselves may store format attributes in any particular manner
they like. Care should be taken by consumers of the API to check the
return value before casting and dereferencing. Consumers will obviously
need to have a priori knowledge of the type of the format attribute as
well.
Change-Id: Ieec76883dfb46ecd7aff3dc81a52c81f4dc1b9e3
Fixes for issues with the ASTERISK-24934 patch.
* Fixed ast_escape_alloc() and ast_escape_c_alloc() if the s parameter is
an empty string. If it were an empty string the functions returned NULL
as if there were a memory allocation failure. This failure caused the AMI
VarSet event to not get posted if the new value was an empty string.
* Fixed dest buffer overwrite potential in ast_escape() and
ast_escape_c(). If the dest buffer size is smaller than the space needed
by the escaped s parameter string then the dest buffer would be written
beyond the end by the nul string terminator. The num parameter was really
the dest buffer size parameter so I renamed it to size.
* Made nul terminate the dest buffer if the source string parameter s was
an empty string in ast_escape() and ast_escape_c().
* Updated ast_escape() and ast_escape_c() doxygen function description
comments to reflect reality.
* Added some more unit test cases to /main/strings/escape to cover the
empty source string issues.
ASTERISK-25255 #close
Reported by: Richard Mudgett
Change-Id: Id77fc704600ebcce81615c1200296f74de254104
This patch adds more tests that exercise the device state API. This includes:
* Tests that cover adding a device state provider, as well as deleting a
device state provider. This also verifies that you cannot add an
already added device state provider, and cannot delete an already
deleted device state provider.
* A test that covers changing device state and receiving said updates
from a device state subscriber. This also covers hitting both the
device state cache as well as a custom device state provider.
* A test that covers converting device state to channel state and device
state values to a string representation and back.
* A test that covers obtaining device state from an active channel and a
channel driver that provides its own device state.
Change-Id: I2adca67ffb405cd8625a5d6df1e3f9b3d945c08d
Because unit tests now want descriptions to not end with a newline, the
sorcery memory cache thrash tests failed to register. This patch
corrects their descriptions.
Change-Id: Id004b1becfdeed8ee3c846f49beab76a5c0f68b6
Gerrit is complaining of conflicts when trying to create a patch series
of all of the cherry-picked master commits, so I have instead squashed
it all into one commit.
ASTERISK-25067 #close
Reported by: Matt Jordan
Change-Id: I6dda90343fae24a75dc5beec84980024e8d61eb9
Analyzing the code shows that the unit test summary and description
strings should not end with a new-line character. Where these strings are
used in the code a new-line is provided for output.
Change-Id: I129284f5e7ca93d82532334076da4c462d3d9fba
Although ast_context_find, ast_context_find_or_create and
ast_context_destroy perform locking of the contexts table,
any context pointer can become invalid at any time that the
contexts table is unlocked. This change adds locking around
all complete operations involving these functions.
Places where ast_context_find was followed by ast_context_destroy
have been replaced with calls ast_context_destroy_by_name.
ASTERISK-25094 #close
Reported by: Corey Farrell
Change-Id: I1866b6787730c9c4f3f836b6133ffe9c820734fa
So this issue is a bit complicated. Since it is possible to pass values to AMI
that contain a '\r\n' (or other similar sequences) these values need to be
escaped. One way to solve this is to escape the values and then pass the escaped
values to the AMI variable parameter string building function. However, this
puts the onus on the pre-build function to escape all string values. This
potentially requires a fair amount of changes along with a lot of string
allocations/freeing for all values.
Surely there is a way to push this complexity down a level into the string
building function itself? This of course is possible, but ends up requiring a
way to distinguish between strings that need to be escaped and those that don't.
The best way to handle this is by introducing a new format specifier in the
format string. For instance a %s (no escape) and %S (escape). However, that is
a bit weird and unexpected.
So faced with those possibilities this patch implements a limited version of the
first option. Instead of attempting to escape all string values this patch only
escapes those values that make sense. This approach limits the number of changes
and doesn't suffer from the odd format specifier problem.
ASTERISK-24934 #close
Reported by: warren smith
Change-Id: Ib55a5b84fe0481b0f2caaaab68c566f392c0aac0
The config wizard was always pulling the first occurrence of
a variable from an ast_variable list but this gets the template
value from the list instead of any overridden value. This patch
creates ast_variable_find_last_in_list() in config.c and updates
res_pjsip_config_wizard to use it instead of
ast_variable_find_in_list. Now the overridden values, where they
exist, are used instead of template variables.
Updated test_config to test the new API.
ASTERISK-25089 #close
Reported-by: George Joseph <george.joseph@fairview5.com>
Tested-by: George Joseph <george.joseph@fairview5.com>
Change-Id: Ifa7ddefc956a463923ee6839dd1ebe021c299de4
These modules save a pointer to the context they create on load, and
use that pointer to destroy the context at unload. It is not safe
to save this pointer, it is replaced during load of pbx_config,
pbx_lua or pbx_ael.
This change causes the modules to pass NULL to ast_context_destroy,
a safer way to perform the unregistration since it does not use
a pointer that could become invalid.
ASTERISK-25085 #close
Reported by: Corey Farrell
Change-Id: I6a00ec8e38046058f97dc703e1adcde9bf517835
Currently you can 'apply' a wizard to an object type but the wizard
always goes at the end of the object type's wizard list. This patch
adds a new ast_sorcery_insert_wizard_mapping function that allows
you to insert a wizard anyplace in the list. I.E. You could
add a caching wizard to an object type and place it before all
wizards.
ast_sorcery_get_wizard_mapping_count and
ast_sorcery_get_wizard_mapping were added to allow examination
of the mapping list.
ast_sorcery_remove_mapping was added to remove a mapping by name.
As part of this patch, the object type's wizard list was converted
from an ao2_container to an AST_VECTOR_RW.
A new test was added to test_sorcery for this capability.
ASTERISK-25044 #close
Change-Id: I9d2469a9296b2698082c0989e25e6848dc403b57
Based on feedback from Corey Farrell and Y Ateya, a few new
macros have been added...
AST_VECTOR_REMOVE which takes a parameter to indicate if
order should be preserved.
AST_VECTOR_ADD_SORTED which adds an element to
a sorted vector.
AST_VECTOR_RESET which cleans all elements from the vector
leaving the storage intact.
Change-Id: I41d32dbdf7137e0557134efeff9f9f1064b58d14
After using the new vector stuff for real I found...
A bug in AST_VECTOR_INSERT_AT that could cause a seg fault.
The callbacks needed to be closer to ao2_callback in behavior
WRT to CMP_MATCH and CMP_STOP behavior and the ability to return
a vector of matched entries.
A pre-existing issue with APPEND and REPLACE was also fixed.
I also added a new macro to test.h that acts like ast_test_validate
but also accepts a return code variable and a cleanup label. As well
as printing the error, it sets the rc variable to AST_TEST_FAIL and
does a goto to the specified label on error. I had a local version
of this in test_vector so I just moved it.
ASTERISK-25045
Change-Id: I05e5e47fd02f61964be13b7e8942bab5d61b29cc
My 13 version of test_vector had an ASTERISK_REGISTER_FILE() macro
call at the top which is only supported in master. Once removed
builds are successful.
Change-Id: I7cac8b669bed6de543bbf4e2eec3cffc9741acdd
Renamed AST_VECTOR_INSERT to AST_VECTOR_REPLACE because it really
does replace not insert. The few users of AST_VECTOR_INSERT were
refactored. Because these are macros, there should be no ABI
compatibility issues.
Added AST_VECTOR_INSERT_AT that actually inserts an element into the
vector at a specific index pushing existing elements to the right.
Added AST_VECTOR_GET_CMP that can retrieve from the vector based
on a user-provided compare function.
Added AST_VECTOR_CALLBACK function that will execute a function
for each element in the vector. Similar to ao2_callback and
ao2_callback_data functions although the vector callback can take
a variable number of arguments. This should allow easy migration
to a vector where a container might be too heavy.
Added read/write locked vector and lock manipulation macros.
Added unit tests.
ASTERISK-25045 #close
Change-Id: I2e07ecc709d2f5f91bcab8904e5e9340609b00e0
When r432935 was merged, it did correctly fix a situation where a FILE read
operation on the middle of a file buffer would not read the requested length
in the parameters passed to the FILE function. Unfortunately, it would also
allow the FILE function to append more bytes than what was available in the
buffer if the length exceeded the end of the buffer length.
This patch takes the minimum of the remaining bytes in the buffer along with
the calculated length to append provided by the original patch, and uses
that as the length to append in the return result. This patch also updates
the unit tests with the scenarios that were originally pointed out in
ASTERISK-21765 that the original implementation treated incorrectly.
ASTERISK-21765
........
Merged revisions 433173 from http://svn.asterisk.org/svn/asterisk/branches/11
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@433174 65c4cc65-6c06-0410-ace0-fbb531ad65f3
I guess nobody uses templates with AST_CONFIG because today if you have a
context that inherits from a template and you call AST_CONFIG on the context,
you'll get the value from the template even if you've overridden it in the
context. This is because AST_CONFIG only gets the first occurrence which is
always from the template.
This patch adds an optional 'index' parameter to AST_CONFIG which lets you
specify the exact occurrence to retrieve, or '-1' to retrieve the last.
The default behavior is the current behavior.
Tested-by: George Joseph
Review: https://reviewboard.asterisk.org/r/4313/
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@430315 65c4cc65-6c06-0410-ace0-fbb531ad65f3
When the publication of attended transfer messages were pushed to another
thread, some subtle race conditions were introduced with the CEL unit tests.
This patch fixes one of them, and pushes the other to ASTERISK-22367, which
already exists to fix another bouncy CEL unit test.
In particular, this patch fixes the test_cel_attended_transfer_bridges_link
test, and defers the test_cel_attended_transfer_bridges_swap test to the
aforementioned JIRA issue.
ASTERISK-22367
........
Merged revisions 428891 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@428892 65c4cc65-6c06-0410-ace0-fbb531ad65f3
When merging the changes up stream in r428687, I missed the fact that the
signature for stasis_message_type_create was changed. This patch fixes
the compilation issues introduced by that merge.
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@428815 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Prior to this patch, all Stasis subscriptions would receive a dedicated
thread for servicing published messages. In contrast, prior to r400178
(see review https://reviewboard.asterisk.org/r/2881/), the subscriptions
shared a thread pool. It was discovered during some initial work on Stasis
that, for a low subscription count with high message throughput, the
threadpool was not as performant as simply having a dedicated thread per
subscriber.
For situations where a subscriber receives a substantial number of messages
and is always present, the model of having a dedicated thread per subscriber
makes sense. While we still have plenty of subscriptions that would follow
this model, e.g., AMI, CDRs, CEL, etc., there are plenty that also fall into
the following two categories:
* Large number of subscriptions, specifically those tied to endpoints/peers.
* Low number of messages. Some subscriptions exist specifically to coordinate
a single message - the subscription is created, a message is published, the
delivery is synchronized, and the subscription is destroyed.
In both of the latter two cases, creating a dedicated thread is wasteful (and
in the case of a large number of peers/endpoints, harmful). In those cases,
having shared delivery threads is far more performant.
This patch adds the ability of a subscriber to Stasis to choose whether or not
their messages are dispatched on a dedicated thread or on a threadpool. The
threadpool is configurable through stasis.conf.
Review: https://reviewboard.asterisk.org/r/4193
ASTERISK-24533 #close
Reported by: xrobau
Tested by: xrobau
........
Merged revisions 428681 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@428687 65c4cc65-6c06-0410-ace0-fbb531ad65f3
As a result of https://reviewboard.asterisk.org/r/3305, res_sorcery_realtime
was tossing database fields that didn't have an exact match to a sorcery
registered field. This broke the ability to use regexes as field names which
manifested itself as a failure of res_pjsip_phoneprov_provider which uses
this capability. It also broke handling of fields that start with '@' in
realtime but I don't think anyone noticed.
This patch does the following...
* Modifies ast_sorcery_fields_register to pre-compile the name regex.
* Modifies ast_sorcery_is_object_field_registered to test the regex if it
exists instead of doing an exact strcmp.
* Modifies res_pjsip_phoneprov_provider with a few tweaks to get it to work
with realtime.
Tested-by: George Joseph
Review: https://reviewboard.asterisk.org/r/4185/
........
Merged revisions 428543 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@428544 65c4cc65-6c06-0410-ace0-fbb531ad65f3
From reviewboard:
"During blind transfer testing, it was noticed that tests were failing
occasionally because the ARI blind transfer event was not being sent.
After investigating, I detected a race condition in the blind transfer
code. When blind transferring a single channel, the actual transfer
operation (i.e. removing the transferee from the bridge and directing
them to the proper dialplan location) is queued onto the transferee
bridge channel. After queuing the transfer operation, the blind transfer
Stasis message is published. At the time of publication, snapshots of
the channels and bridge involved are created. The ARI subscriber to the
blind transfer Stasis message then attempts to determine if the bridge
or any of the involved channels are subscribed to by ARI applications.
If so, then the blind transfer message is sent to the applications. The
way that the ARI blind transfer message handler works is to first see
if the transferer channel is subscribed to. If not, then iterate over
all the channel IDs in the bridge snapshot and determine if any of
those are subscribed to. In the test we were running, the lone
transferee channel was subscribed to, so an ARI event should have been
sent to our application. Occasionally, though, the bridge snapshot did
not have any channels IDs on it at all. Why?
The problem is that since the blind transfer operation is handled by a
separate thread, it is possible that the transfer will have completed and
the channels removed from the bridge before we publish the blind transfer
Stasis message. Since the blind transfer has completed, the bridge on
which the transfer occurred no longer has any channels on it, so the
resulting bridge snapshot has no channels on it. Through investigation of
the code, I found that attended transfers can have this issue too for the
case where a transferee is transferred to an application."
The fix employed here is to decouple the creation of snapshots for the transfer
messages from the publication of the transfer messages. This way, snapshots
can be created to reflect what they are at the time of the transfer operation.
Review: https://reviewboard.asterisk.org/r/4135
........
Merged revisions 427848 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@427870 65c4cc65-6c06-0410-ace0-fbb531ad65f3
When a config file is read, an unescaped semicolon signals comments which are
stripped from the value before it's stored. Escaped semicolons are then
unescaped and become part of the value. Both of these behaviors are normal
and expected. When the config is serialized either by 'dialplan save' or
AMI/UpdateConfig however, the now unescaped semicolons are written as-is.
If you actually reload the file just saved, the unescaped semicolons are
now treated as start of comments.
Since true comments are stripped on read, any semicolons in
ast_variable.value must have been escaped originally. This patch
re-escapes semicolons in ast_variable.values before they're written to
file either by 'dialplan save' or config/ast_config_text_file_save which
is called by AMI/UpdateConfig. I also fixed a few pre-existing formatting
issues nearby in pbx_config.c
Tested-by: George Joseph
ASTERISK-20127 #close
Review: https://reviewboard.asterisk.org/r/4132/
........
Merged revisions 427275 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@427276 65c4cc65-6c06-0410-ace0-fbb531ad65f3
The CEL pickup test previously looked for a disposition of ANSWER between the
original caller/peer when the call is picked up. This is actually incorrect:
the disposition should, at the very least, not be ANSWER as the call was
never ANSWERed. The disposition is now CANCEL; this patch updates the test
accordingly.
........
Merged revisions 425757 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@425758 65c4cc65-6c06-0410-ace0-fbb531ad65f3
With MALLOC_DEBUG the /main/config config_basic_ops test was causing a
SEGV while doing an ast_category_delete in an ast_category_browse loop.
Apparently this never worked but was also never tested. I removed the
test, added 2 notes to config.h indicating that it's not supported and
added a few lines of code to ast_category_delete to prevent the SEGV
should someone attempt it in the future.
Tested-by: George Joseph
Review: https://reviewboard.asterisk.org/r/4078/
........
Merged revisions 425525 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@425526 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This patch provides the capability to manipulate templates and categories
with non-unique names via AMI.
Summary of changes:
GetConfig and GetConfigJSON: Added "Filter" parameter: A comma separated list
of name_regex=value_regex expressions which will cause only categories whose
variables match all expressions to be considered. The special variable name
TEMPLATES can be used to control whether templates are included. Passing
'include' as the value will include templates along with normal categories.
Passing 'restrict' as the value will restrict the operation to ONLY templates.
Not specifying a TEMPLATES expression results in the current default behavior
which is to not include templates.
UpdateConfig: NewCat now includes options for allowing duplicate category
names, indicating if the category should be created as a template, and
specifying templates the category should inherit from. The rest of the
actions now accept a filter string as defined above. If there are non-unique
category names, you can now update specific ones based on variable values.
To facilitate the new capabilities in manager, corresponding changes had to be
made to config, most notably the addition of filter criteria to many of the
APIs. In some cases it was easy to change the references to use the new
prototype but others would have required touching too many files for this
patch so a wrapper with the original prototype was created. Macros couldn't
be used in this case because it would break binary compatibility with modules
such as res_digium_phone that are linked to real symbols.
Tested-by: George Joseph
Review: https://reviewboard.asterisk.org/r/4033/
........
Merged revisions 425383 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@425384 65c4cc65-6c06-0410-ace0-fbb531ad65f3
Prior to this commit, CDR and CEL tests were expected to trigger
FRACKs (i.e. assertions) due to the fact that the channels they
create have no formats on them. Some code was independently added
recently that attempts to prevent FRACKs from occurring by failing
early when attempting to set up translation paths if one or both
channels support no formats. Unfortunately, this attempt to be helpful
made the CDR and CEL tests go from simply FRACKing to outright
failing and in some cases, failing so badly as to crash Asterisk.
This commit seeks to correct past mistakes by adding the ulaw format
to channels created by the CDR and CEL unit tests. This makes setting
up translation paths succeed, eliminates previously-seen FRACKs, and
ultimately causes the unit tests to succeed again.
Review: https://reviewboard.asterisk.org/r/4014
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@423783 65c4cc65-6c06-0410-ace0-fbb531ad65f3
This function acts like strsep with three exceptions...
* The separator is a single character instead of a string.
* Separators inside quotes are treated literally instead of like separators.
* You can elect to have leading and trailing whitespace and quotes
stripped from the result and have '\' sequences unescaped.
Like strsep, ast_strsep maintains no internal state and you can call it
recursively using different separators on the same storage.
Also like strsep, for consistent results, consecutive separators are not
collapsed so you may get an empty string as a valid result.
Tested by: George Joseph
Review: https://reviewboard.asterisk.org/r/3989/
........
Merged revisions 423476 from http://svn.asterisk.org/svn/asterisk/branches/12
git-svn-id: https://origsvn.digium.com/svn/asterisk/branches/13@423478 65c4cc65-6c06-0410-ace0-fbb531ad65f3
