asterisk/doc/tex/app-sms.tex

\section{Introduction}

   The SMS module for Asterisk was developed by Adrian Kennard, and is an
   implementation of the ETSI specification for landline SMS, ETSI ES 201
   912, which is available from \url{www.etsi.org}. Landline SMS is starting to
   be available in various parts of Europe, and is available from BT in
   the UK. However, Asterisk would allow gateways to be created in other
   locations such as the US, and use of SMS capable phones such as the
   Magic Messenger. SMS works using analogue or ISDN lines.

\section{Background}

   Short Message Service (SMS), or texting is very popular between mobile
   phones. A message can be sent between two phones, and normally
   contains 160 characters. There are ways in which various types of data
   can be encoded in a text message such as ring tones, and small
   graphic, etc. Text messaging is being used for voting and
   competitions, and also SPAM...

   Sending a message involves the mobile phone contacting a message
   centre (SMSC) and passing the message to it. The message centre then
   contacts the destination mobile to deliver the message. The SMSC is
   responsible for storing the message and trying to send it until the
   destination mobile is available, or a timeout.

   Landline SMS works in basically the same way. You would normally have
   a suitable text capable landline phone, or a separate texting box such
   as a Magic Messenger on your phone line. This sends a message to a
   message centre your telco provides by making a normal call and sending
   the data using 1200 Baud FSK signaling according to the ETSI spec. To
   receive a message the message centre calls the line with a specific
   calling number, and the text capable phone answers the call and
   receives the data using 1200 Baud FSK signaling. This works
   particularly well in the UK as the calling line identity is sent
   before the first ring, so no phones in the house would ring when a
   message arrives.

\section{Typical use with Asterisk}

   Sending messages from an Asterisk box can be used for a variety of
   reasons, including notification from any monitoring systems, email
   subject lines, etc.

   Receiving messages to an Asterisk box is typically used just to email
   the messages to someone appropriate - we email and texts that are
   received to our direct numbers to the appropriate person. Received
   messages could also be used to control applications, manage
   competitions, votes, post items to IRC, anything.

   Using a terminal such as a magic messenger, an Asterisk box could ask
   as a message centre sending messages to the terminal, which will beep
   and pop up the message (and remember 100 or so messages in its
   memory).

\section{Terminology}

\begin{itemize}
   \item SMS -
   Short Message Service
   i.e. text messages

   \item SMSC -
   Short Message Service Centre
   The system responsible for storing and forwarding messages

   \item MO -
   Mobile Originated
   A message on its way from a mobile or landline device to the SMSC

   \item MT -
   Mobile Terminated
   A message on its way from the SMSC to the mobile or landline device

   \item RX -
   Receive
   A message coming in to the Asterisk box

   \item TX -
   Transmit
   A message going out of the Asterisk box
\end{itemize}

\section{Sub address}

   When sending a message to a landline, you simply send to the landline
   number. In the UK, all of the mobile operators (bar one) understand
   sending messages to landlines and pass the messages to the BTText
   system for delivery to the landline.

   The specification for landline SMS allows for the possibility of more
   than one device on a single landline. These can be configured with Sub
   addresses which are a single digit. To send a message to a specific
   device the message is sent to the landline number with an extra digit
   appended to the end. The telco can define a default sub address (9 in
   the UK) which is used when the extra digit is not appended to the end.
   When the call comes in, part of the calling line ID is the sub
   address, so that only one device on the line answers the call and
   receives the message.

   Sub addresses also work for outgoing messages. Part of the number
   called by the device to send a message is its sub address. Sending
   from the default sub address (9 in the UK) means the message is
   delivered with the sender being the normal landline number. Sending
   from any other sub address makes the sender the landline number with
   an extra digit on the end.

   Using Asterisk, you can make use of the sub addresses for sending and
   receiving messages. Using DDI (DID, i.e. multiple numbers on the line
   on ISDN) you can also make use of many different numbers for SMS.

\section{extensions.conf}

   The following contexts are recommended.

\begin{astlisting}
\begin{verbatim}
; Mobile Terminated, RX. This is used when an incoming call from the SMS arrive
s, with the queue (called number and sub address) in ${EXTEN}
; Running an app after receipt of the text allows the app to find all messages
in the queue and handle them, e.g. email them.
; The app may be something like   smsq --process=somecommand --queue=${EXTEN}
to run a command for each received message
; See below for usage
[smsmtrx]
exten = _X.,1, SMS(${EXTEN},a)
exten = _X.,2,System("someapptohandleincomingsms ${EXTEN}")
exten = _X.,3,Hangup
; Mobile originated, RX. This is receiving a message from a device, e.g.
; a Magic Messenger on a sip extension
; Running an app after receipt of the text allows the app to find all messages
; in the queue and handle then, e.g. sending them to the public SMSC
; The app may be something like   smsq --process=somecommand --queue=${EXTEN}
; to run a command for each received message
; See below for example usage
[smsmorx]
exten = _X.,1, SMS(${EXTEN},sa)
exten = _X.,2,System("someapptohandlelocalsms ${EXTEN}")
exten = _X.,3,Hangup
\end{verbatim}
\end{astlisting}

   smsmtrx is normally accessed by an incoming call from the SMSC. In the
   UK this call is from a CLI of 080058752X0 where X is the sub address.
   As such a typical usage in the extensions.conf at the point of
   handling an incoming call is:
\begin{astlisting}
\begin{verbatim}
exten = _X./8005875290,1,Goto(smsmtrx,${EXTEN},1)
exten = _X./_80058752[0-8]0,1,Goto(smsmtrx,${EXTEN}-${CALLERID(num):8:1},1)
\end{verbatim}
\end{astlisting}

   Alternatively, if you have the correct national prefix on incoming
   CLI, e.g. using dahdi\_hfc, you might use:
\begin{astlisting}
\begin{verbatim}
exten = _X./08005875290,1,Goto(smsmtrx,${EXTEN},1)
exten = _X./_080058752[0-8]0,1,Goto(smsmtrx,${EXTEN}-${CALLERID(num):9:1},1)
\end{verbatim}
\end{astlisting}

   smsmorx is normally accessed by a call from a local sip device
   connected to a Magic Messenger. It could however by that you are
   operating Asterisk as a message centre for calls from outside. Either
   way, you look at the called number and goto smsmorx. In the UK, the
   SMSC number that would be dialed is 1709400X where X is the caller sub
   address. As such typical usage in extension.config at the point of
   handling a call from a sip phone is:
\begin{astlisting}
\begin{verbatim}
exten = 17094009,1,Goto(smsmorx,${CALLERID(num)},1)
exten = _1709400[0-8],1,Goto(smsmorx,${CALLERID(num)}-{EXTEN:7:1},1)
\end{verbatim}
\end{astlisting}

\section{Using smsq}

   smsq is a simple helper application designed to make it easy to send
   messages from a command line. it is intended to run on the Asterisk
   box and have direct access to the queue directories for SMS and for
   Asterisk.

   In its simplest form you can send an SMS by a command such as
   smsq 0123456789 This is a test to 0123456789
   This would create a queue file for a mobile originated TX message in
   queue 0 to send the text "This is a test to 0123456789" to 0123456789.
   It would then place a file in the \path{/var/spool/asterisk/outgoing}
   directory to initiate a call to 17094009 (the default message centre
   in smsq) attached to application SMS with argument of the queue name
   (0).

   Normally smsq will queue a message ready to send, and will then create
   a file in the Asterisk outgoing directory causing Asterisk to actually
   connect to the message centre or device and actually send the pending
   message(s).

   Using \verb!--process!, smsq can however be used on received queues to run a
   command for each file (matching the queue if specified) with various
   environment variables set based on the message (see below);
   smsq options:
\begin{verbatim}
   --help
   Show help text
   --usage
   Show usage
   --queue
   -q
   Specify a specific queue
   In no specified, messages are queued under queue "0"
   --da
   -d
   Specify destination address
   --oa
   -o
   Specify originating address
   This also implies that we are generating a mobile terminated message
   --ud
   -m
   Specify the actual message
   --ud-file
   -f
   Specify a file to be read for the context of the message
   A blank filename (e.g. --ud-file= on its own) means read stdin. Very
   useful when using via ssh where command line parsing could mess up the
   message.
   --mt
   -t
   Mobile terminated message to be generated
   --mo
   Mobile originated message to be generated
   Default
   --tx
   Transmit message
   Default
   --rx
   -r
   Generate a message in the receive queue
   --UTF-8
   Treat the file as UTF-8 encoded (default)
   --UCS-1
   Treat the file as raw 8 bit UCS-1 data, not UTF-8 encoded
   --UCS-2
   Treat the file as raw 16 bit bigendian USC-2 data
   --process
   Specific a command to process for each file in the queue
   Implies --rx and --mt if not otherwise specified.
   Sets environment variables for every possible variable, and also ud,
   ud8 (USC-1 hex), and ud16 (USC-2 hex) for each call. Removes files.
   --motx-channel
   Specify the channel for motx calls
   May contain X to use sub address based on queue name or may be full
   number
   Default is Local/1709400X
   --motx-callerid
   Specify the caller ID for motx calls
   The default is the queue name without -X suffix
   --motx-wait
   Wait time for motx call
   Default 10
   --motx-delay
   Retry time for motx call
   Default 1
   --motx-retries
   Retries for motx call
   Default 10
   --mttx-channel
   Specify the channel for mttx calls
   Default is Local/ and the queue name without -X suffix
   --mtttx-callerid
   Specify the callerid for mttx calls
   May include X to use sub address based on queue name or may be full
   number
   Default is 080058752X0
   --mttx-wait
   Wait time for mttx call
   Default 10
   --mttx-delay
   Retry time for mttx call
   Default 30
   --mttx-retries
   Retries for mttx call
   Default 100
   --default-sub-address
   The default sub address assumed (e.g. for X in CLI and dialled numbers
   as above) when none added (-X) to queue
   Default 9
   --no-dial
   -x
   Create queue, but do not dial to send message
   --no-wait
   Do not wait if a call appears to be in progress
   This could have a small window where a message is queued but not
   sent, so regular calls to smsq should be done to pick up any missed
   messages
   --concurrent
   How many concurrent calls to allow (per queue), default 1
   --mr
   -n
   Message reference
   --pid
   -p
   Protocol ID
   --dcs
   Data coding scheme
   --udh
   Specific hex string of user data header specified (not including the
   initial length byte)
   May be a blank string to indicate header is included in the user data
   already but user data header indication to be set.
   --srr
   Status report requested
   --rp
   Return path requested
   --vp
   Specify validity period (seconds)
   --scts
   Specify timestamp (YYYY-MM-DDTHH:MM:SS)
   --spool-dir
   Spool dir (in which sms and outgoing are found)
   Default /var/spool/asterisk
\end{verbatim}

   Other arguments starting '-' or '\verb!--!' are invalid and will cause an
   error. Any trailing arguments are processed as follows:-

\begin{itemize}

     \item If the message is mobile originating and no destination address
       has been specified, then the first argument is assumed to be a
       destination address

     \item If the message is mobile terminating and no destination address
       has been specified, then the first argument is assumed to be the
       queue name

     \item If there is no user data, or user data file specified, then any
       following arguments are assumed to be the message, which are
       concatenated.

     \item If no user data is specified, then no message is sent. However,
       unless \verb!--no-dial! is specified, smsq checks for pending messages
       and generates an outgoing anyway
\end{itemize}


   Note that when smsq attempts to make a file in
   \path{/var/spool/asterisk/outgoing}, it checks if there is already a call
   queued for that queue. It will try several filenames, up to the
   \verb!--concurrent! setting. If these files exist, then this means Asterisk
   is already queued to send all messages for that queue, and so Asterisk
   should pick up the message just queued. However, this alone could
   create a race condition, so if the files exist then smsq will wait up
   to 3 seconds to confirm it still exists or if the queued messages have
   been sent already. The \verb!--no-wait! turns off this behaviour. Basically,
   this means that if you have a lot of messages to send all at once,
   Asterisk will not make unlimited concurrent calls to the same message
   centre or device for the same queue. This is because it is generally
   more efficient to make one call and send all of the messages one after
   the other.

   smsq can be used with no arguments, or with a queue name only, and it
   will check for any pending messages and cause an outgoing if there are
   any. It only sets up one outgoing call at a time based on the first
   queued message it finds. A outgoing call will normally send all queued
   messages for that queue. One way to use smsq would be to run with no
   queue name (so any queue) every minute or every few seconds to send
   pending message. This is not normally necessary unless \verb!--no-dial! is
   selected. Note that smsq does only check motx or mttx depending on the
   options selected, so it would need to be called twice as a general
   check.

   UTF-8 is used to parse command line arguments for user data, and is
   the default when reading a file. If an invalid UTF-8 sequence is
   found, it is treated as UCS-1 data (i.e, as is).
   The \verb!--process! option causes smsq to scan the specified queue (default
   is mtrx) for messages (matching the queue specified, or any if queue
   not specified) and run a command and delete the file. The command is
   run with a number of environment variables set as follows. Note that
   these are unset if not needed and not just taken from the calling
   environment. This allows simple processing of incoming messages
\begin{verbatim}
   $queue
   Set if a queue specified
   $?srr
   srr is set (to blank) if srr defined and has value 1.
   $?rp
   rp is set (to blank) if rp defined and has value 1.
   $ud
   User data, UTF-8 encoding, including any control characters, but with
   nulls stripped out
   Useful for the content of emails, for example, as it includes any
   newlines, etc.
   $ude
   User data, escaped UTF-8, including all characters, but control
   characters \n, \r, \t, \f, \xxx and \ is escaped as \\
   Useful guaranteed one line printable text, so useful in Subject lines
   of emails, etc
   $ud8
   Hex UCS-1 coding of user data (2 hex digits per character)
   Present only if all user data is in range U+0000 to U+00FF
   $ud16
   Hex UCS-2 coding of user data (4 hex digits per character)
   other
   Other fields set using their field name, e.g. mr, pid, dcs, etc. udh
   is a hex byte string
\end{verbatim}

\section{File formats}

   By default all queues are held in a director \path{/var/spool/asterisk/sms}.
   Within this directory are sub directories mtrx, mttx, morx, motx which
   hold the received messages and the messages ready to send. Also,
   \path{/var/log/asterisk/sms} is a log file of all messages handled.

   The file name in each queue directory starts with the queue parameter
   to SMS which is normally the CLI used for an outgoing message or the
   called number on an incoming message, and may have -X (X being sub
   address) appended. If no queue ID is known, then 0 is used by smsq by
   default. After this is a dot, and then any text. Files are scanned for
   matching queue ID and a dot at the start. This means temporary files
   being created can be given a different name not starting with a queue
   (we recommend a . on the start of the file name for temp files).
   Files in these queues are in the form of a simple text file where each
   line starts with a keyword and an = and then data. udh and ud have
   options for hex encoding, see below.

   UTF-8. The user data (ud) field is treated as being UTF-8 encoded
   unless the DCS is specified indicating 8 bit format. If 8 bit format
   is specified then the user data is sent as is.
   The keywords are as follows:
\begin{verbatim}
   oa Originating address
   The phone number from which the message came
   Present on mobile terminated messages and is the CLI for morx messages
   da
   Destination Address
   The phone number to which the message is sent
   Present on mobile originated messages
   scts
   The service centre time stamp
   Format YYYY-MM-DDTHH:MM:SS
   Present on mobile terminated messages
   pid
   One byte decimal protocol ID
   See GSM specs for more details
   Normally 0 or absent
   dcs
   One byte decimal data coding scheme
   If omitted, a sensible default is used (see below)
   See GSM specs for more details
   mr
   One byte decimal message reference
   Present on mobile originated messages, added by default if absent
   srr
   0 or 1 for status report request
   Does not work in UK yet, not implemented in app_sms yet
   rp
   0 or 1 return path
   See GSM specs for details
   vp
   Validity period in seconds
   Does not work in UK yet
   udh
   Hex string of user data header prepended to the SMS contents,
   excluding initial length byte.
   Consistent with ud, this is specified as udh# rather than udh=
   If blank, this means that the udhi flag will be set but any user data
   header must be in the ud field
   ud
   User data, may be text, or hex, see below
\end{verbatim}

   udh is specified as as udh\# followed by hex (2 hex digits per byte).
   If present, then the user data header indicator bit is set, and the
   length plus the user data header is added to the start of the user
   data, with padding if necessary (to septet boundary in 7 bit format).
   User data can hold an USC character codes U+0000 to U+FFFF. Any other
   characters are coded as U+FEFF

   ud can be specified as ud= followed by UTF-8 encoded text if it
   contains no control characters, i.e. only (U+0020 to U+FFFF). Any
   invalid UTF-8 sequences are treated as is (U+0080-U+00FF).

   ud can also be specified as ud\# followed by hex (2 hex digits per
   byte) containing characters U+0000 to U+00FF only.

   ud can also be specified as ud\#\# followed by hex (4 hex digits per
   byte) containing UCS-2 characters.

   When written by app\_sms (e.g. incoming messages), the file is written
   with ud= if it can be (no control characters). If it cannot, the a
   comment line ;ud= is used to show the user data for human readability
   and ud\# or ud\#\# is used.

\section{Delivery reports}

   The SMS specification allows for delivery reports. These are requested
   using the srr bit. However, as these do not work in the UK yet they
   are not fully implemented in this application. If anyone has a telco
   that does implement these, please let me know. BT in the UK have a non
   standard way to do this by starting the message with *0\#, and so this
   application may have a UK specific bodge in the near future to handle
   these.

   The main changes that are proposed for delivery report handling are :

\begin{itemize}
     \item New queues for sent messages, one file for each destination
       address and message reference.

     \item New field in message format, user reference, allowing applications
       to tie up their original message with a report.

     \item Handling of the delivery confirmation/rejection and connecting to
       the outgoing message - the received message file would then have
       fields for the original outgoing message and user reference
       allowing applications to handle confirmations better.
\end{itemize}