Asterisk by default will create a single database connection and share
it among all threads that attempt to access the database. In previous
versions of Asterisk, this was tolerable, because the most used channel
driver, chan_sip, mostly accessed the database from a single thread.
With PJSIP, however, many threads may be attempting to perform database
operations, and there is the potential for many more database accesses,
meaning the concurrency is a horrible bottleneck if only one connection
is shared.
Asterisk has a connection pooling facility built into it, but the
implementation has flaws. For one, there is a strict limit on the number
of simultaneous connections that could be made to the database. Anything
beyond the maximum would result in a failed operation. Attempting to
predict what the maximum should be is nearly impossible even for someone
intimately familiar with Asterisk's threading model. In addition, use of
transactions in the dialplan can cause some severe bugs if connection
pooling is enabled.
This commit seeks to fix the concurrency problem by removing all
connection management code from Asterisk and leaving that to the
underlying unixODBC code instead. Now, Asterisk does not share a single
connection, nor does it try to maintain a connection pool. Instead, all
Asterisk ever does is request a connection from unixODBC and allow
unixODBC to either allocate those connections or retrieve them from a
pool.
Doing this has a bit of a ripple effect. For one, since connections are
not long-lived objects, several of the safeguards that previously
existed have been removed. We don't have to worry about trying to use a
connection that has gone stale. In every case, when we request a
connection, it has just been made and we don't need to perform any
sanity checks to be sure it's still active.
Another major player affected by this change is transactions.
Transactions and their respective connections were so tightly coupled
that it was almost pornographic. This code change moves
transaction-related code to its own file separate from the core ODBC
functionality. This way, the core of ODBC does not even have to know
that transactions exist.
In making this large change, I had to look at a lot of code and
understand it. When making this change, I discovered several places
where the behavior is definitely not ideal, but it seemed outside the
scope of this change to be fixing it. Instead, any place where I saw
some sort of room for improvement has had a XXX comment added explaining
what could be altered to improve it.
Change-Id: I37a84def5ea4ddf93868ce8105f39de078297fbf
Dump the res_pjsip endpt internals.
In non-developer mode we will not document or make easily accessible the
"details" option even though it is still available. The user has to know
it exists to use it. Presumably they would also be aware of the potential
crash warning below.
Warning: PJPROJECT documents that the function used by this CLI command
may cause a crash when asking for details because it tries to access all
active memory pools.
Change-Id: If2d98a3641c9873364d1daaad971376311aef3cb
res_pjsip_log_forwarder has been renamed to res_pjproject
and enhanced as follows:
As a follow-on to the recent 'Add CLI "pjsip show buildopts"' patch,
a new ast_pjproject_get_buildopt function has been added. It
allows the caller to get the value of one of the buildopts.
The initial use case is retrieving the runtime value of
PJ_MAX_HOSTNAME to insure we don't send a hostname greater
than pjproject can handle. Since it can differ between
the version of pjproject that Asterisk was compiled against
and the version of pjproject that Asterisk is running against,
we can't use the PJ_MAX_HOSTNAME macro directly in Asterisk
source code.
Change-Id: Iab6e82fec3d7cf00c1cf6185c42be3e7569dee1e
The xferfailsound was read from the channel at the beginning of the transfer,
and that value is "cached" for the duration of the transfer. Therefore, changing
the xferfailsound on the channel using the FEATURE() dialplan function does
nothing once the transfer is under way.
This makes it so the transfer code instead gets the xferfailsound configuration
options from the channel when it is actually going to be used.
This patch also fixes a potential memory leak of the props object as well as
making sure the condition variable gets initialized before being destroyed.
ASTERISK-25696 #close
Change-Id: Ic726b0f54ef588bd9c9c67f4b0e4d787934f85e4
* changes:
Sorcery: Create human friendly serializer names.
Stasis: Create human friendly taskprocessor/serializer names.
taskprocessor.c: New API for human friendly taskprocessor names.
taskprocessor.c: Sort CLI "core show taskprocessors" output.
On a system with multiple ip addresses in the same subnet, if a
transport is bound to a specific ip address and endpoint/media_address
is set, the SIP/SDP will have the correct address in all fields but
the rtp stream MAY still originate from one of the other ip addresses,
most probably the "primary" ip address. This happens because
res_pjsip_sdp_rtp/create_rtp always calls ast_instance_new with
the "all" ip address (0.0.0.0 or ::).
The new option causes res_pjsip_sdp_rtp/create_rtp to call
ast_rtp_instance_new with the endpoint's media_address (if specified)
instead of the "all" address. This causes the packets to originate from
the specified address.
ASTERISK-25632
ASTERISK-25637
Reported-by: Olivier Krief
Reported-by: Dan Journo
Change-Id: I3dfaa079e54ba7fb7c4fd1f5f7bd9509bbf8bd88
Added new global option (regcontext) to pjsip. When set, Asterisk will
dynamically create and destroy a NoOp priority 1 extension
for a given endpoint who registers or unregisters with us.
ASTERISK-25670 #close
Reported-by: Daniel Journo
Change-Id: Ib1530c5b45340625805c057f8ff1fb240a43ea62
* Add new API call to get a sequence number for use in human friendly
taskprocessor names.
* Add new API call to create a taskprocessor name in a given buffer and
append a sequence number.
Change-Id: Iac458f05b45232315ed64aa31b1df05b875537a9
Renamed global declaration:tv to dummy_tv_var_for_types,
which would oltherwise cause 'shadow' warnings when 'tv'
was declared as a local variable elsewhere.
Added comment to note that dummy_tv_var_for_types is never
really exported and only used as a place holder.
ASTERISK-25627 #close
Change-Id: I9a6e17995006584f3627efe8988e3f8aa0f5dc28
The 11/13 branches and master use 2 different file version macros. 11/13
uses ASTERISK_FILE_VERSION but master uses ASTERISK_REGISTER_FILE. This
means a new file added to 11/13 can't just be cherry-picked to master
because the macro has to be changed.
To make cherry-picking possible, ASTERISK_REGISTER_FILE was added
to asterisk.h as a simple alias for ASTERISK_FILE_VERSION(__FILE__, NULL)
The "$Revision$" tag doesn't do anything since Asterisk moved to git so
just passing NULL as the verison works fine. asterisk.h was also
annotated to deprecate ASTERISK_FILE_VERSION and suggest using
ASTERISK_REGISTER_FILE for all new files.
Finally, 2 recent file additions, pbx_builtins.c and pbx_functions.c,
were modified to use the new macro to make sure it actually worked.
'core show file version' showed the correct output.
Change-Id: I5867ed898818d26ee49bb6e5c7d4c1a45d4789a5
Somehow stasis_cache_pattern got out of sync between 13 and master
and it was causing duplicate channel message issues in 13 when
related to a specific endpoint. I.E. from statsd,
'endpoints.PJSIP.1174.channels 0|g' was being emitted twice.
Backporting stasis_cache_pattern from master to 13 solved
the issue and running the unit and testsuite tests confirmed
that no new ones were created.
ASTERISK-25317 #close
Change-Id: Ia8707462f62d15eed14541c37f332a7bbbceb548
This is the sixth patch in a series meant to reduce the bulk of pbx.c.
This moves hangup handler management functions to their own source.
Change-Id: Ib25a75aa57fc7d5c4294479e5cc46775912fb104
This is the sixth patch in a series meant to reduce the bulk of pbx.c.
This moves dialplan application management functions to their own source.
Change-Id: I444c10fb90a3cdf9f3047605d6a8aad49c22c44c
This is the fifth patch in a series meant to reduce the bulk of pbx.c.
This moves ast_switch functions to their own source.
Change-Id: Ic2592a18a5c4d8a3c2dcf9786c9a6f650a8c628e
The menuselect conflict between app_voicemail and res_mwi_external
makes it hard to package 1 version of Asterisk. There no actual
build dependencies between the 2 so moving this check to runtime
seems like a better solution.
The ast_vm_register and ast_vm_greeter_register functions in app.c
were modified to return AST_MODULE_LOAD_DECLINE instead of -1 if there
is already a voicemail module registered. The modules' load_module
functions were then modified to return DECLINE instead of -1 to the
loader. Since -1 is interpreted by the loader as AST_MODULE_LOAD_FAILURE,
the modules were incorrectly causing Asterisk to stop so this needed
to be cleaned up anyway.
Now you can build both and use modules.conf to decide which voicemail
implementation to load.
The default menuselect options still build app_voicemail and not
res_mwi_external but if both ARE built, res_mwi_external will load
first and become the voicemail provider unless modules.conf rules
prevent it. This is noted in CHANGES.
Change-Id: I7d98d4e8a3b87b8df9e51c2608f0da6ddfb89247
This is the third patch in a series meant to reduce the bulk of pbx.c.
This moves channel and global variable routines to their own source.
Change-Id: Ibe8fb4647db11598591d443a99e3f99200a56bc6
This is the second patch in a series meant to reduce the bulk of pbx.c.
This moves custom function management routines to their own source.
Change-Id: I34a6190282f781cdbbd3ce9d3adeac3c3805e177
We joked about splitting pbx.c into multiple files but this first step was
fairly easy. All of the pbx_builtin dialplan applications have been moved
into pbx_builtins.c and a new pbx_private.h file was added. load_pbx_builtins()
is called by asterisk.c just after load_pbx().
A few functions were renamed and are cross-exposed between the 2 source files.
Change-Id: I87066be3dbf7f5822942ac1449d98cc43fc7561a
When an endpoint is created, its messages are forwarded to both the tech
endpoint topic and the all endpoints topic. This is done so that various
parties interested in endpoint messages can subscribe to just the tech
endpoint and receive all messages associated with that particular technology,
as opposed to subscribing to the all endpoints topic. Unfortunately, when the
tech endpoint is created, it also forwards all of its messages to the all
topic. This results in duplicate messages whenever an endpoint publishes its
messages.
This patch resolves the duplicate message issue by creating a new function
for Stasis caching topics, stasis_cp_sink_create. In most respects, this acts
as a normal caching topic, save that it no longer forwards messages it receives
to the all endpoints topic. This allows it to act as an aggregation "sink",
while preserving the necessary caching behaviour.
ASTERISK-25137 #close
Reported-by: Vitezslav Novy
ASTERISK-25116 #close
Reported-by: George Joseph <george.joseph@fairview5.com>
Tested-by: George Joseph <george.joseph@fairview5.com>
Change-Id: Ie47784adfb973ab0063e59fc18f390d7dd26d17b
Updated ast_websocket_write to encode the entire frame in to one
write operation, to ensure that we don't end up with a situation
where the websocket header has been sent, while the body can not
be written.
Previous to August's patch in commit b9bd3c14, certain network
conditions could cause the header to be written, and then the
sub-sequent body to fail - which would cause the next successful
write to contain a new header, and a new body (resulting in
the peer receiving two headers - the second of which would be
read as part of the body for the first header).
This was patched to have both write operations individually fail
by closing the websocket.
In a case available to the submitter of this patch, the same
body which would consistently fail to write, would succeed
if written at the same time as the header.
This update merges the two operations in to one, adds debug messages
indicating the reason for a websocket connection being closed during
a write operation, and clarifies some variable names for code legibility.
Change-Id: I4db7a586af1c7a57184c31d3d55bf146f1a40598
pjproject < 2.5.0 will segfault on a tls transport if async_operations
is greater than 1. A runtime version check has been added to throw
an error if the version is < 2.5.0 and async_operations > 1.
To assist in the check, a new api "ast_compare_versions" was added
to utils which compares 2 major.minor.patch.extra version strings.
ASTERISK-25615 #close
Change-Id: I8e88bb49cbcfbca88d9de705496d6f6a8c938a98
Reported-by: George Joseph
Tested-by: George Joseph
Both transport and endpoint now check for the existence and readability
of tls certificate and key files before passing them on to pjproject.
This will cause the object to not load rather than waiting for pjproject
to discover that there's a problem when a session is attempted.
NOTE: chan_sip also uses ast_rtp_dtls_cfg_parse but it's located
in build_peer which is gigantic and I didn't want to disturb it.
Error messages will emit but it won't interrupt chan_sip loading.
ASTERISK-25618 #close
Change-Id: Ie43f2c1d653ac1fda6a6f6faecb7c2ebadaf47c9
Reported-by: George Joseph
Tested-by: George Joseph
An earlier commit changed the id of dynamic contacts to contain
a hash instead of the uri. This patch updates status change
logging to show the aor/uri instead of the id. This required
adding the aor id to contact and contact_status and adding
uri to contact_status. The aor id gets added to contact and
contact_status in their allocators and the uri gets added to
contact_status in pjsip_options when the contact_status is
created or updated.
ASTERISK-25598 #close
Reported-by: George Joseph
Tested-by: George Joseph
Change-Id: I56cbec1d2ddbe8461367dd8b6da8a6f47f6fe511
Currently if a channel is transferred out of a bridge, the BRIDGEPEER
variable (also BRIDGEPVTCALLID) remain set even once the channel is
out of the bridge. This patch removes these variables when leaving
the bridge.
ASTERISK-25600 #close
Reported by: Mark Michelson
Change-Id: I753ead2fffbfc65427ed4e9244c7066610e546da
Several issues are addressed here:
- main() is large, and half of it is only used if we're not rasterisk;
fixed by spliting up the daemon part into a separate function.
- Call ast_term_init from rasterisk as well.
- Remove duplicate code reading/writing asterisk history file.
- Attempt to tackle background color issues and color changes that
occur. Tested by starting asterisk -c until the colors stopped
changing at odd locations.
ASTERISK-25585 #close
Change-Id: Ib641a0964c59ef9fe6f59efa8ccb481a9580c52f
Previously, a trancoding module did not have access to the joint but cached
format. Therefore, the module did not have access to the attributes negotiated
via SDP (line fmtp). Now, a translation module receives the joint format.
ASTERISK-25545 #close
Change-Id: Id6878a989b50573298dab115d3371ea369e1a718
Often, the metric names of statistics we are generating for StatsD have some
dynamic component to them. This can be the name of a particular resource, or
some internal status label in Asterisk. With the current set of functions,
callers of the statsd API must first build the metric name themselves, then
pass this to the API functions. This results in a large amount of boilerplate
code and usage of either fixed length static buffers or dynamic memory
allocation, neither of which is desireable.
This patch adds two new functions to the StatsD API that support a printf
style format specifier for constructing the metric name. A dynamic string,
allocated in threadstorage, is used to build the metric name. This eases
the burden on users of the StatsD API.
Change-Id: If533c72d1afa26d807508ea48b4d8c7b32f414ea
Added a new api to res_statsd.c to allow it to receive a
character pointer for the value argument. This allows for a
'+' and a '-' to easily be sent with the value.
ASTERISK-25419
Reported By: Ashley Sanders
Change-Id: Id6bb53600943d27347d2bcae26c0bd5643567611
In practical tests, we have seen certain taskprocessors, specifically
Stasis subscription taskprocessors, cross the recently-added high-water
mark and emit a warning. This high-water mark warning is only intended
to be emitted when things have tanked on the system and things are
heading south quickly. In the practical tests, the Stasis taskprocessors
sometimes had a max depth of 180 tasks in them, and Asterisk wasn't in
any danger at all.
As such, this ups the high-water mark to 500 tasks instead. It also
redefines the SIP threadpool request denial number to be a multiple of
the taskprocessor high-water mark.
Change-Id: Ic8d3e9497452fecd768ac427bb6f58aa616eebce
We have observed situations where the SIP threadpool may become
deadlocked. However, because incoming traffic is still arriving, the SIP
threadpool's queue can continue to grow, eventually running the system
out of memory.
This change makes it so that incoming traffic gets rejected with a 503
response if the queue is backed up too much.
Change-Id: I4e736d48a2ba79fd1f8056c0dcd330e38e6a3816
A previous commit reduced the AST_BUILDOPTS compiler define to
only include options that affected ABI. This included some options
that were previously displayed by cli "core show settings". This
change corrects the CLI display while still restricting buildopts.h
to ABI effecting options only.
ASTERISK-25434 #close
Reported by: Rusty Newton
Change-Id: Id07af6bedd1d7d325878023e403fbd9d3607e325
Add the ability to filter output from pjsip list and show commands
using the "like" predicate like chan_sip.
For endpoints, aors, auths, registrations, identifyies and transports,
the modification was a simple change of an ast_sorcery_retrieve_by_fields
call to ast_sorcery_retrieve_by_regex. For channels and contacts a
little more work had to be done because neither of those objects are
true sorcery objects. That was just removing the non-matching object
from the final container. Of course, a little extra plumbing in the
common pjsip_cli code was needed to parse the "like" and pass the regex
to the get_container callbacks.
Some of the get_container code in res_pjsip_endpoint_identifier was also
refactored for simplicity.
ASTERISK-25477 #close
Reported by: Bryant Zimmerman
Tested by: George Joseph
Change-Id: I646d9326b778aac26bb3e2bcd7fa1346d24434f1
There have been crashes and general instability seen in the pubsub code,
so this patch introduces three changes to increase the stability.
First, the ownership model for subscriptions has been modified. Due to
RLS, subscriptions are stored in memory as a tree structure. Prior to my
patch, the PJSIP subscription was the owner of the subscription tree.
When the PJSIP subscription told us that it was terminating, we started
destroying the subscription tree along with all of the individual leaf
subscriptions that belong to the tree. The problem with this model is
that the two actors in play here, the PJSIP subscription and the
individual leaf subscriptions, need to have joint ownership of the
subscription tree. So now, the PJSIP subscription and the individual
leaf subscriptions each have a reference to the subscription tree. This
way, we will not actually free memory until no players are left that
care. The PJSIP subscription is a bigger stakeholder, in that if the
PJSIP subscription's reference to the subscription tree is removed, the
subscription tree instructs the leaf subscriptions to shut down and drop
their references to the subscription tree when possible. The individual
leaf subscriptions, upon being told to shut down, can drop their stasis
subscriptions or whatever they use to learn of new state, and then drop
their reference to the subscription tree once they are ready to die.
Second, the lifetime of a PJSIP subscription's reference to our
subscription tree has been altered. As I learned from doing a deep dive,
the PJSIP evsub code can tell Asterisk multiple times that the
subscription has been terminated, and not all of these times
are especially helpful. I have altered the message flow that we use for
SIP subscriptions such that we will always drop the PJSIP subscription's
reference to the subscription tree when we send the NOTIFY that
terminates a SIP subscription. This also means that we will now queue
NOTIFY requests to be sent after responding to incoming SUBSCRIBEs so
that we can have predictable state changes from the PJSIP evsub code.
Third, the synchronization of operations has been improved. PJSIP can
call into our code from a serializer thread (e.g. upon receiving an
incoming request) or from the monitor thread (e.g. when a subscription
times out). Because of this, there is the possibility of competing
threads stepping on each other. PJSIP attempts to do some
synchronization on its own by always keeping the dialog lock held when
it calls into us. However, since we end up pushing tasks into the
serializer, the result was that serialized operations were not grabbing
the dialog lock and could, as a result, step on something that was being
attempted by a different thread. Now we ensure that serialized
operations grab the dialog lock, then check for extenuating
circumstances, then proceed with their operation if they can.
Change-Id: Iff2990c40178dad9cc5f6a5c7f76932ec644b2e5
In a realtime based system with a limited number of threadpool threads
it is possible for a deadlock to occur. This happens when permanent
endpoint state is updated, which will cause database queries to be done.
These queries may result in URI validation being done which is done
synchronously using a PJSIP thread. If all PJSIP threads are in use
processing traffic they themselves may be blocked waiting to get the
permanent endpoint container lock when identifying an endpoint.
This change moves URI validation to occur at use time instead of
configuration time. While this comes at a cost of not seeing a problem
until you use it it does solve the underlying deadlock problem.
ASTERISK-25486 #close
Change-Id: I2d7d167af987d23b3e8199e4a68f3359eba4c76a
This patch adds the ability to subscribe to all events. There are two possible
ways to accomplish this:
(1) On initial WebSocket connection. This patch adds a new query parameter,
'subscribeAll'. If present and True, Asterisk will subscribe the
applications to all ARI events.
(2) Via the applications resource. When subscribing in this manner, an ARI
client should merely specify a blank resource name, i.e., 'channels:'
instead of 'channels:12354'. This will subscribe the application to all
resources of the 'channels' type.
ASTERISK-24870 #close
Change-Id: I4a943b4db24442cf28bc64b24bfd541249790ad6
