There were a number of places in the res_pjsip stack that were getting
all endpoints or all aors, and then filtering them locally.
A good example is pjsip_options which, on startup, retrieves all
endpoints, then the aors for those endpoints, then tests the aors to see
if the qualify_frequency is > 0. One issue was that it never did
anything with the endpoints other than retrieve the aors so we probably
could have skipped a step and just retrieved all aors. But nevermind.
This worked reasonably well with local config files but with a realtime
backend and thousands of objects, this was a nightmare. The issue
really boiled down to the fact that while realtime supports predicates
that are passed to the database engine, the non-realtime sorcery
backends didn't.
They do now.
The realtime engines have a scheme for doing simple comparisons. They
take in an ast_variable (or list) for matching, and the name of each
variable can contain an operator. For instance, a name of
"qualify_frequency >" and a value of "0" would create a SQL predicate
that looks like "where qualify_frequency > '0'". If there's no operator
after the name, the engines add an '=' so a simple name of
"qualify_frequency" and a value of "10" would return exact matches.
The non-realtime backends decide whether to include an object in a
result set by calling ast_sorcery_changeset_create on every object in
the internal container. However, ast_sorcery_changeset_create only does
exact string matches though so a name of "qualify_frequency >" and a
value of "0" returns nothing because the literal "qualify_frequency >"
doesn't match any name in the objset set.
So, the real task was to create a generic string matcher that can take a
left value, operator and a right value and perform the match. To that
end, strings.c has a new ast_strings_match(left, operator, right)
function. Left and right are the strings to operate on and the operator
can be a string containing any of the following: = (or NULL or ""), !=,
>, >=, <, <=, like or regex. If the operator is like or regex, the
right string should be a %-pattern or a regex expression. If both left
and right can be converted to float, then a numeric comparison is
performed, otherwise a string comparison is performed.
To use this new function on ast_variables, 2 new functions were added to
config.c. One that compares 2 ast_variables, and one that compares 2
ast_variable lists. The former is useful when you want to compare 2
ast_variables that happen to be in a list but don't want to traverse the
list. The latter will traverse the right list and return true if all
the variables in it match the left list.
Now, the backends' fields_cmp functions call ast_variable_lists_match
instead of ast_sorcery_changeset_create and they can now process the
same syntax as the realtime engines. The realtime backend just passes
the variable list unaltered to the engine. The only gotcha is that
there's no common realtime engine support for regex so that's been noted
in the api docs for ast_sorcery_retrieve_by_fields.
Only one more change to sorcery was done... A new config flag
"allow_unqualified_fetch" was added to reg_sorcery_realtime.
"no": ignore fetches if no predicate fields were supplied.
"error": same as no but emit an error. (good for testing)
"yes": allow (the default);
"warn": allow but emit a warning. (good for testing)
Now on to res_pjsip...
pjsip_options was modified to retrieve aors with qualify_frequency > 0
rather than all endpoints then all aors. Not only was this a big
improvement in realtime retrieval but even for config files there's an
improvement because we're not going through endpoints anymore.
res_pjsip_mwi was modified to retieve only endpoints with something in
the mailboxes field instead of all endpoints then testing mailboxes.
res_pjsip_registrar_expire was completely refactored. It was retrieving
all contacts then setting up scheduler entries to check for expiration.
Now, it's a single thread (like keepalive) that periodically retrieves
only contacts whose expiration time is < now and deletes them. A new
contact_expiration_check_interval was added to global with a default of
30 seconds.
Ross Beer reports that with this patch, his Asterisk startup time dropped
from around an hour to under 30 seconds.
There are still objects that can't be filtered at the database like
identifies, transports, and registrations. These are not going to be
anywhere near as numerous as endpoints, aors, auths, contacts however.
Back to allow_unqualified_fetch. If this is set to yes and you have a
very large number of objects in the database, the pjsip CLI commands
will attempt to retrive ALL of them if not qualified with a LIKE.
Worse, if you type "pjsip show endpoint <tab>" guess what's going to
happen? :) Having a cache helps but all the objects will have to be
retrieved at least once to fill the cache. Setting
allow_unqualified_fetch=no prevents the mass retrieve and should be used
on endpoints, auths, aors, and contacts. It should NOT be used for
identifies, registrations and transports since these MUST be
retrieved in bulk.
Example sorcery.conf:
[res_pjsip]
endpoint=config,pjsip.conf,criteria=type=endpoint
endpoint=realtime,ps_endpoints,allow_unqualified_fetch=error
ASTERISK-25826 #close
Reported-by: Ross Beer
Tested-by: Ross Beer
Change-Id: Id2691e447db90892890036e663aaf907b2dc1c67
The configuration unsigned integer option handler sets flags for the
parser as if the option should be a signed integer (PARSE_INT32),
leading to errors on "out of range" values. Fix flags (PARSE_UINT32).
A fix to res_pjsip is also present which stops invalid flags from
being passed when registering sorcery object fields for qualify
status.
ASTERISK-25612 #close
Change-Id: I96b539336275e0e72a8e8033487d2c3344debd3e
During stress testing, we have frequently seen crashes occur because a
CLI or AMI command attempts to access information that is in the process
of being destroyed.
When addressing how to fix this issue, we initially considered fixing
individual crashes we observed. However, the changes required to fix
those problems would introduce considerable overhead to the nominal
case. This is not reasonable in order to prevent a crash from occurring
while Asterisk is already shutting down.
Instead, this change makes it so AMI and CLI commands cannot be executed
if Asterisk is being shut down. For AMI, this is absolute. For CLI,
though, certain commands can be registered so that they may be run
during Asterisk shutdown.
ASTERISK-25825 #close
Change-Id: I8887e215ac352fadf7f4c1e082da9089b1421990
The log message when a MusicOnHold music class was not found was changed
from debug level to WARNING level in Asterisk 11.19 and 13.5. For those
using realtime musiconhold, this message is wrong because it warns
before checking the database.
This changeset delays the warning until after the database has been
checked.
Reported-by: Conrad de Wet
ASTERISK-25444 #close
Change-Id: I6cfb2db2f9cfbd2bb3d30566ecae361c4abf6dbf
The fix to ASTERISK-25407 introduced the usage of LOG_MAKEPRI. However
this macro is broken in older glibc (< 2.17); it would left-shift the
facility a second time, causing the resultant priority to become
invalid.
The syslog manpage mentions nothing about LOG_MAKEPRI and suggests this:
The priority argument is formed by ORing the facility and the level
values [...].
ASTERISK-25510 #close
Reported by: Michael Newton
Change-Id: Ia89debe7fac5ad090c7ef595c0707f31bb1e3d03
PortAudio should no longer be required
PJSIP_MAX_PKT_LEN is now 6000
Older autoconf issue fixed. (CentOS 6)
Change-Id: I463fa9586cbe7c6b3b603289f535bd8e361611dd
check_via() already skips leading blanks where the sent-by address (with the
optional port) should be placed.
Since RFC 3261 allows for blanks between the port ant the Via parameters:
> https://tools.ietf.org/html/rfc3261#section-20.42
(actually it allows a lot of blanks more ;-)). I just switched from
ast_skip_blanks() to ast_strip() on the local copy of the string.
ASTERISK-21301 #close
Change-Id: Ie5b8fe5a07067b7c0dc9bcdd1707e99b23b02b06
The encryption code for AES_ENCRYPT evaluates the length of the data to
be encoded in base64 using strlen. The data is binary, thus the length
of it can be underestimated at the first NULL character.
Reuse the write pointer offset to evaluate it, instead.
ASTERISK-25857 #close
Change-Id: If686b5d570473eb926693c73461177b35b13b186
During a transfer involving direct media a race occurs between when the
transferer channel is swapped out, initiating rtp changes/updates, and the
subsequent reinvites.
When Alice, after speaking with Charlie (Bob is on hold), connects Bob and
Charlie invites are sent to each in order to establish the call between them.
Bob is taken off hold and Charlie is told to have his media flow through
Asterisk. However, if before those invites go out the bridge updates Bob's
and/or Charlie's rtp information with direct media data (i.e. address, port)
then the invite(s) will contain the remote data in the SDP instead of the
Asterisk data.
The race occurs in the native bridge glue code when updating the peer. The
direct_media_address can get set twice before sending out the first invite
during call connection. This can happen because the checking/setting of the
direct_media_address happened in one thread while the sending of the invite(s)
happened in another thread.
This fix removes the race condition by moving the checking/setting of the
direct_media_address to be in the same thread as the sending of the invites(s).
This serializes the checking/setting and sending so they can no longer happen
out of order.
ASTERISK-25849 #close
Change-Id: Idfea590175e74f401929a601dba0c91ca1a7f873
When to install packages the indexed local is more old of the
version of software on the repository they have been upgraded by security
update then get the package will give 404 not found.
The patch prevent by update local index to repository for aptitude before
install.
ASTERISK-25495 #close
Reporte by: Rodrigo Ramírez Norambuena
Change-Id: I645959e553aac542805ced394cac2dca964051fa
(cherry picked from commit 88f3dbaec9)
If in Debian or system based, dont have aptitude installed the script do
nothing. This patch checked if aptitude installed, if not installed.
Also, if execute script with all packages installed yet, the script not show
nothing and return exit 1 because the command 'grep' get nothing from pipe from
'awk'.
ASTERISK-25113 #close
Reported By: Rodrigo Ramírez Norambuena <decipher.hk@gmail.com>
Change-Id: Iebdff55805d3917166e5e08e0a1e2176f36ff27f
(cherry picked from commit 6737ded058)
The "Refer-to" header of an incoming REFER request is parsed by
pjsip_parse_uri(). That function requires the URI parameter to be NULL
terminated. Unfortunately, the previous code added the NULL terminator by
overwriting memory that may not be safe. The overwritten memory results
could be benign, memory corruption, or a segmentation fault. Now the URI
is NULL terminated safely by copying the URI to a new chunk of memory with
the correct size to be NULL terminated.
ASTERISK-25814 #close
Change-Id: I32565496684a5a49c3278fce06474b8c94b37342
This patch is part of a series to resolve deadlocks in chan_sip.c.
Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event. If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen. The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event. Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.
ASTERISK-25023 #close
Change-Id: I96d429c57a48861fd8bde63dd93db4e92dc3adb6
This patch is part of a series to resolve deadlocks in chan_sip.c.
Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event. If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen. The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event. Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.
ASTERISK-25023
Change-Id: I2e40de89efc8ae6e8850771d089ca44bc604b508
This patch is part of a series to resolve deadlocks in chan_sip.c.
Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event. If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen. The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event. Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.
ASTERISK-25023
Change-Id: If595e4456cd059d7171880c7f354e844c21b5f5f
This patch is part of a series to resolve deadlocks in chan_sip.c.
Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event. If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen. The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event. Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.
ASTERISK-25023
Change-Id: I9c11b9d597468f63916c99e1dabff9f4a46f84c1
This patch is part of a series to resolve deadlocks in chan_sip.c.
Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event. If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen. The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event. Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.
* Fix retrans_pkt() to call check_pendings() with both the owner channel
and the private objects locked as required.
* Refactor dialog retransmission packet list to safely remove packet
nodes. The list nodes are now ao2 objects. The list has a ref and the
scheduled entry has a ref.
ASTERISK-25023
Change-Id: I50926d81be53f4cd3d572a3292cd25f563f59641
This patch is part of a series to resolve deadlocks in chan_sip.c.
Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event. If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen. The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event. Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.
* Made always run check_pendings() under the scheduler thread so scheduler
ids can be checked safely.
ASTERISK-25023
Change-Id: Ia834d6edd5bdb47c163e4ecf884428a4a8b17d52
This patch is part of a series to resolve deadlocks in chan_sip.c.
Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event. If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen. The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event. Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.
ASTERISK-25023
Change-Id: I6d65269151ba95e0d8fe4e9e611881cde2ab4900
This patch is part of a series to resolve deadlocks in chan_sip.c.
Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event. If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen. The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event. Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.
* Fix clearing autokillid in __sip_autodestruct() even though we could
reschedule.
ASTERISK-25023
Change-Id: I450580dbf26e2e3952ee6628c735b001565c368f
This patch is part of a series to resolve deadlocks in chan_sip.c.
Stopping a scheduled event can result in a deadlock if the scheduled event
is running when you try to stop the event. If you hold a lock needed by
the scheduled event while trying to stop the scheduled event then a
deadlock can happen. The general strategy for resolving the deadlock
potential is to push the actual starting and stopping of the scheduled
events off onto the scheduler/do_monitor() thread by scheduling an
immediate one shot scheduled event. Some restructuring may be needed
because the code may assume that the start/stop of the scheduled events is
immediate.
ASTERISK-25023
Change-Id: I98a694fd42bc81436c83aa92de03226e6e4e3f48
This patch is part of a series to resolve deadlocks in chan_sip.c.
* Make dialog_unlink_all() unschedule all items at once in the sched
thread.
ASTERISK-25023
Change-Id: I7743072fb228836e8228b72f6dc46c8cc50b3fb4
This patch is part of a series to resolve deadlocks in chan_sip.c.
The reordering of chan_sip's shutdown is to handle any immediate events
that get put onto the scheduler so resources aren't leaked. The typical
immediate events at this time are going to be concerned with stopping
other scheduled events.
ASTERISK-25023
Change-Id: I3f6540717634f6f2e84d8531a054976f2bbb9d20
This patch is part of a series to resolve deadlocks in chan_sip.c.
Delaying destruction of the chan_sip sip_pvt structures caused the
/channels/chan_sip/test_sip_rtpqos unit test to crash. That test
registers a special test ast_rtp_engine with the rtp engine module. When
the unit test completes it cleans up by unregistering the test
ast_rtp_engine and exits. Since the delayed destruction of the sip_pvt
happens after the unit test returns, the destructor tries to call the rtp
engine destroy callback of the test ast_rtp_engine auto variable which no
longer exists on the stack.
* Change the test ast_rtp_engine auto variable to a static variable. Now
the variable can still exist after the unit test exits so the delayed
sip_pvt destruction can complete successfully.
ASTERISK-25023
Change-Id: I61e34a12d425189ef7e96fc69ae14993f82f3f13
This prevents pbx_core from hanging up the channel if the app isn't
registered.
ASTERISK-25846 #close
Change-Id: I63216a61f30706d5362bc0906b50b6f0544aebce
This patch is part of a series to resolve deadlocks in chan_sip.c.
* Updated sched unit test to check new behavior.
ASTERISK-25023
Change-Id: Ib69437327b3cda5e14c4238d9ff91b2531b34ef3
Remove destructor calling destroy_it calling really_destroy_it
for no benefit. Just make the destructor the really_destroy_it
function.
Change-Id: Idea0d47b27dd74f2488db75bcc7f353d8fdc614a
Older versions of PJSIP do not have the proto field on the TLS transport
setting structure. This change adds a configure check so even if it is
not present we will still be able to build.
Change-Id: Ibf3f47befb91ed1b8194bf63888baa6fee05aba9
I can't ever recall actually needing the intermediate files or the checking
that a double compile produces. What I CAN remember is every DONT_OPTIMIZE
build needing 3 invocations of gcc instead of 1 just to do the checks and
produce those intermediate files.
Having said that, Richard pointed out that the reason for the double compile
was that there were cases in the past where a submitted patch failed to compile
because the submitter never tried it with the optimizations turned on.
To get the best of both worlds, COMPILE_DOUBLE has been split into its own
option. If DONT_OPTIMIZE is turned on, COMPILE_DOUBLE will also be selected
BUT you can then turn it off if all you need are the debugging symbols. This
way you have to make an informed decision about disabling COMPILE_DOUBLE.
To allow COMPILE_DOUBLE to be both auto-selected and turned off, a new feature
was added to menuselect. The <use> element can now contain an "autoselect"
attribute which will turn the used member on but not create a hard dependency.
The cflags.xml implementation for COMPILE_DOUBLE looks like this...
<member name="DONT_OPTIMIZE" displayname="Disable Optimizations ...">
<use autoselect="yes">COMPILE_DOUBLE</use>
<support_level>core</support_level>
</member>
<member name="COMPILE_DOUBLE" displayname="Pre-compile with ...>
<depend>DONT_OPTIMIZE</depend>
<support_level>core</support_level>
</member>
When DONT_OPTIMIZE is turned on, COMPILE_DOUBLE is turned on because
of the use.
When DONT_OPTIMIZE is turned off, COMPILE_DOUBLE is turned off because
of the depend.
When COMPILE_DOUBLE is turned on, DONT_OPTIMIZE is turned on because
of the depend.
When COMPILE_DOUBLE is turned off, DONT_OPTIMIZE is left as is because
it only uses COMPILE_DOUBLE, it doesn't depend on it.
I also made a few tweaks to the ncurses implementation to move things
left a bit to allow longer descriptions.
Change-Id: Id49ca930ac4b5ec4fc2d8141979ad888da7b1611
