Howto speed up agi in asterisk with inetd
Asterisk's AGI interface performance issues can be a limiting factor in high performance installations. This post details how we used agi+inetd for a 4x improvement in calls per second (cps) with minimal change in our agi code
Test setup
Amazon ec2's high cpu medium instance (the one that costs USD0.17/hr) + Slackware 13.0 + asterisk 1.6.2.7 as the OwnPages call handling server. Another such instance was used for sipp to generate calls, and 2 such were used with Vicidial + asterisk 1.4.x as dummy SIP terminators. We used Vici's excellent performance test that gave approximately our production ACD, ASR and failure cause stats out of the box.
Before agi+inetd
Our existing Asterisk installations for OwnPages choke at 7cps for server side call handling, which we assumed was due to faulty agi design and our use of 1.4.x versions of asterisk. So our first step was to switch (for the test machine) to 1.6.2.7 and inspect our ancient agi processes (written in c) for code and database inefficiencies, of which we found many. To our surprise, the cps meter barely moved. See first row in table of test results
We then began to systematically remove functionality from the agi processes to isolate functions that were resource intensive. In stages we stripped the agi processes to bare shells that did virtually nothing, and found that it made almost no difference to cps.
Next step was to replace our agis completely with a simple call to /bin/ls for a directory that had just a few files. When we saw cps stuck at 7 even for this, we understood that the act of calling the agi processes mattered more than what the processes did.
Our diaplan had 3 agi processes per call attempt. We combined the functionality into just one agi process per call, and saw an immediate 2x improvement to 15cps :-)
Enter inetd
15 calls per second gave us about 350 concurrent calls without media for our call stats. See second row in table below. This was much less than our goal of 1000 concurrent calls for optimum manageability. We then set about removing this last agi call.
Inetd (with tcpd) implements a network wrapper around a normal process by presenting network input to its standard input, and similarly handling its output. The process itself needs no network code. Asterisk 1.6's Fastagi interface is capable of sending variables to the network connection. All we had to do to use Fastagi instead of agi was to modify the agi to accept its variables from command line (for use with agi API) as well as standard input (to use with Fastagi+inetd). This was a simple change in our agi code. We then called our agi from /etc/inetd.conf, effectively using inetd as a Fastagi server.
We liked this change. CPS went from 15 to 32, and concurrent calls without media touched 800. See third row in table below. We were still short of our goal, but figured we could more than get there with Amazon's high CPU extra large instance type which gives you 4x CPU (at 4x cost). We have not tried it because it is a 64bit instance type. We had managed our goal with no drop in functionality.
Test results
| Max cps | Max concurrent calls | Server load avg | CPU idle % | |
| Before inetd, 3agis/call | 7 | 175 | 4 | 10 |
| Before inetd, 1 combined agi/call | 15 | 350 | 6 | 14 |
| After inetd, 1 combined agi/call | 32 | 800 | 6.8 | 22 |
While the table seems to suggest that the act of combining agis was as important as switching to inetd, our testing indicates that just switching to inetd should give you 3.5 - 4x improvement in cases where agi performance is the limiting factor. We did not measure this clearly, since we had already combined the agi processes by the time we got to inetd