WWW.DUMAIS.IO
ARTICLES
OVERLAY NETWORKS WITH MY SDN CONTROLLERSIMPLE LEARNING SWITCH WITH OPENFLOWINSTALLING KUBERNETES MANUALLYWRITING A HYPERVISOR WITH INTEL VT-X CREATING YOUR OWN LINUX CONTAINERSVIRTIO DRIVER IMPLEMENTATIONNETWORKING IN MY OSESP8266 BASED IRRIGATION CONTROLLERLED STRIP CONTROLLER USING ESP8266.OPENVSWITCH ON SLACKWARESHA256 ASSEMBLY IMPLEMENTATIONPROCESS CONTEXT ID AND THE TLBTHREAD MANAGEMENT IN MY HOBBY OSENABLING MULTI-PROCESSORS IN MY HOBBY OSNEW HOME AUTOMATION SYSTEMINSTALLING AND USING DOCKER ON SLACKWARESYSTEM ON A CHIP EMULATORUSING JSSIP AND ASTERISK TO MAKE A WEBPHONEC++ WEBSOCKET SERVERSIP ATTACK BANNINGBLOCK CACHING AND WRITEBACKBEAGLEBONE BLACK BARE METAL DEVELOPEMENTARM BARE METAL DEVELOPMENTUSING EPOLLMEMORY PAGINGIMPLEMENTING HTTP DIGEST AUTHENTICATIONSTACK FRAME AND THE RED ZONE (X86_64)AVX/SSE AND CONTEXT SWITCHINGHOW TO ANSWER A QUESTION THE SMART WAY.REALTEK 8139 NETWORK CARD DRIVERREST INTERFACE ENGINECISCO 1760 AS AN FXS GATEWAYHOME AUTOMATION SYSTEMEZFLORA IRRIGATION SYSTEMSUMP PUMP MONITORINGBUILDING A HOSTED MAILSERVER SERVICEI AM NOW HOSTING MY OWN DNS AND MAIL SERVERS ON AMAZON EC2DEPLOYING A LAYER3 SWITCH ON MY NETWORKACD SERVER WITH RESIPROCATEC++ JSON LIBRARYIMPLEMENTING YOUR OWN MUTEX WITH CMPXCHGWAKEUPCALL SERVER USING RESIPROCATEFFT ON AMD64CLONING A HARD DRIVECONFIGURING AND USING KVM-QEMUUSING COUCHDBINSTALLING COUCHDB ON SLACKWARENGW100 MY OS AND EDXS/LSENGW100 - MY OSASTERISK FILTER APPLICATIONCISCO ROUTER CONFIGURATIONAASTRA 411 XML APPLICATIONSPA941 PHONEBOOKSPEEDTOUCH 780 DOCUMENTATIONAASTRA CONTACT LIST XML APPLICATIONAVR32 OS FOR NGW100ASTERISK SOUND INJECTION APPLICATIONNGW100 - DIFFERENT PROBLEMS AND SOLUTIONSAASTRA PRIME RATE XML APPLICATIONSPEEDTOUCH 780 CONFIGURATIONUSING COUCHDB WITH PHPAVR32 ASSEMBLY TIPAP7000 AND NGW100 ARCHITECTUREAASTRA WEATHER XML APPLICATIONNGW100 - GETTING STARTEDAASTRA ALI XML APPLICATION

I AM NOW HOSTING MY OWN DNS AND MAIL SERVERS ON AMAZON EC2

2012-09-22


Using an Amazon EC2 virtual machine, I wanted to host my own domain by doing as much as I could myself. By that, I mean that I wanted to host my own DNS server, mail server and web server. So after registering a new domain name, the first thing I did was to change the nameserver of that domain to use my own server instead of the registrar's. After that, all the work is done on my EC2 machine. Note that for this to work, I needed to unblock TCP port 25 and 53 and also UDP port 53 on my virtual server. This can be done in the EC2 management console under the "security groups" section.

Configuring the DNS server

First thing to do is to add the new zone definition in /etc/named.conf. Let's assume that I registered "exampledomain.ca".

zone "exampledomain.ca" IN { type master; file "exampledomain.ca"; };

Then I need to define the zone in /var/named/exampledomain.ca. I wanted to create a subdomain called "www" so that my web server could be accessible from www.exampledomain.ca. I needed an MX record that points to the mail server that is also hosted on the same server as the DNS server. For this example, let's assume that the public IP address of my server is 1.2.3.4.

$TTL 300 @ IN SOA exampledomain.ca. www.exampledomain.ca. ( 2012220104 ; 1H ; refresh 15M ; retry 1W ; expiry 1D ) ; minimum IN NS ns. IN A 1.2.3.4 IN MX 10 mail ns IN A 1.2.3.4 www IN A 1.2.3.4 mail IN A 1.2.3.4

Mail server

after looking for a solution to host multiple domains with virtual mailboxes that are not associated with a unix account, I discovered that that postix might be better for the job than sendmail. after installing postfix and setting it to run automatically at boot time, we need to configure the /etc/postfix/main.cf file. I am only telling it to listen to all network interface and where to look for the list of domains we are hosting and the mailbox definition.

The other nice thing about postfix VS sendmail is that you can easily add more mailboxes/forwards and domains without having to restart postfix. This is very usefull if you wanna make a webpage on which users can create their own mailboxes and add their own domains. But the nicest thing of all is that you can use a mysql server (or several other kinds of DB) so that your web site can configure the server more easily. But for now, I will stick with scripts. Here is an example of my main.cf script:

queue_directory = /var/spool/postfix command_directory = /usr/sbin daemon_directory = /usr/libexec/postfix data_directory = /var/lib/postfix mail_owner = postfix # listen to all interfaces so we can accept incomming connection from LAN and WAN. inet_interfaces = all inet_protocols = all mydestination = $myhostname, localhost.$mydomain, localhost unknown_local_recipient_reject_code = 550 alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases debug_peer_level = 2 debugger_command = PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin ddd $daemon_directory/$process_name $process_id & sleep 5 sendmail_path = /usr/sbin/sendmail.postfix newaliases_path = /usr/bin/newaliases.postfix mailq_path = /usr/bin/mailq.postfix setgid_group = postdrop html_directory = no manpage_directory = /usr/share/man sample_directory = /usr/share/doc/postfix-2.6.6/samples readme_directory = /usr/share/doc/postfix-2.6.6/README_FILES # mailboxes will be saved in /var/vhosts virtual_mailbox_base = /var/vhosts #/etc/postfix/vmbox will contain the list of mailboxes virtual_mailbox_maps = hash:/etc/postfix/vmbox #/etc/postfix/vdomains will contain the list of domains we are accepting mail for. virtual_mailbox_domains = hash:/etc/postfix/vdomains virtual_alias_maps = hash:/etc/postfix/valias # The /var/vhosts folder must be writeable by user 4000 and group 4000 virtual_uid_maps = static:4000 virtual_gid_maps = static:4000

A note on postifx lookup tables. Postifx is great because it lets use use different type of lookup tables. In the example posted above, I use hash:/etc/postfix/valias. This is a reference to a Berkely DB file. You need to create a text file with key/pair values separated by a space and generate a DB with the "postmap" command in order to use this. But if you want to use another table type such as a mysql table, then you could specify mysql:/etc/postfix/valias. But in that case, the "valias" file's content would be different. It would contain information on how to access the database table.

More information on different types of tables can be found here: http://www.postfix.org/DATABASE_README.html

The virtual_uid_maps and virtual_gid_maps lines indicate that the mailboxes inside /var/vhosts will be owned by 4000:4000 on the system. This means that you have to create a user with ID 4000 and change ownership of the base folder to that user:group

useradd virtualmailbox -u 4000 chown virtualmailbox:virtualmailbox /var/vhosts

The virtual_mailbox_domains line indicates that postfix needs to look into that file to find the list of domains it is accepting mail for. Everytime that you modify the vdomains of vmbox file, you need to rebuild the hash database by issuing the "postmap /etc/postfix/vdomains" command. Of course, you would replace "vdomains" with "vmbox" if you need to rebuild the vmbox database. virtual mailboxes are files that will be stored on the server. But if you need to forward to another address, then you will need aliases. I'm not sure why, but I discovered that you should not use "virtual_alias_domains" if you want to use mailboxes and aliases for the same domains. Only list the domains in "virtual_mailbox_maps".

# This file will be compiled by the "postmap" tool. It needs # to contain 2 columns on each line. Since this is just a list # of domains, we only need one column but the tool will expect 2. # So you can put anything you want in the second column. I decided # to put the domain name a second time. # exampledomain.ca exampledomain.ca

vmbox file

#because there is a slash at the end of the path, mail will be saved in maildir format admin@exampledomain.ca exampledomain.ca/admin/ #because there is no slash at the end of the path, mail will be saved in linux mbox format test@exampledomain.ca exampledomain.ca/test

valias file

user1@exampledomain.ca user1@gmail.com user2@exampledomain.ca user2@yahoo.com

Automation

It could become very handy to automate the process of adding domains and user mailbox though a perl of php script. The best way to do it would be by using a database, but let's see how to do it with files first.

Files

  • Add the domain name in /etc/postfix/vdomains
  • Add mailbox in /etc/postfix/vmbox
  • Add forwarding addresses in /etc/postfix/valias
  • Run "postmap /etc/postfix/vdomains"
  • Run "postmap /etc/postfix/vmbox"
  • Run "postmap /etc/postfix/valias"

Of course, this requires that the script locks the files to prevent two process from accessing the same files at the same time. And also, it would be better to schedule the "postmap" instead or running it everytime because if you have a large system where users create several mailboxes in 1 minute, you would want to wait a little bit before running postmap oherwise you would add a great load on your CPU.

Database

With this method, your scripts don't need to lock any files and you don't need to schedule a "postmap". All you need to do is to insert a mailbox/domain in the database and the change is instantaneous. In my opinion, this is the ideal way of doing thing if you have a server that hosts several domains and you allow users to create their own mailboxes. For an example on how to do this, you can refer to this post I did: Building a hosted mailserver service

DEPLOYING A LAYER3 SWITCH ON MY NETWORK

2012-08-20

Lately, I bought a Cisco 3550 switch with 48 ports. My goal was to have a switch that could do the inter-vlan routing instead of relying on the router. This way, if my router goes down, I only lose my internet connection but the voice network and data network on my LAN can still function properly. I was also using the cisco router as my DHCP server, but now it is running on my switch.

The first thing that needs to be configured is the SVI interface. SVI stands for Switch Virtual Interface. It represents a interface to a vlan. You need one SVI for each VLAN that you want to provide routing on. By creating an SVI, you assign it an IP address. This IP address is the address that you will use as the default gateway on all nodes on your VLAN.

! create SVI for VLAN 1 interface Vlan1 ip address 192.168.1.1 255.255.255.0 ! create SVI for VLAN 2 interface Vlan2 ip address 192.168.2.1 255.255.255.0

Since I wanna host my DHCP server on the switch now instead of the router, I can take the same configuration I had on the router at apply it on the switch.

ip dhcp excluded-address 192.168.1.1 192.168.1.99 ip dhcp excluded-address 192.168.1.150 192.168.1.255 ! ip dhcp pool pool_vlan1 import all network 192.168.1.0 255.255.255.0 default-router 192.168.1.1 dns-server 192.168.254.1 ! ip dhcp pool pool_vlan2 import all network 192.168.2.0 255.255.255.0 default-router 192.168.2.1 dns-server 192.168.254.1

Notice that the DNS server (forwarder) is on the router and not on the switch. Usually, the DNS server in my setups was on the same device as the subnet's gateway. So on all the devices that were assigned a static IP on my network, I had to change the IP address of the DNS server.

Connecting the router and the switch

There are two ways of doing this: SVI or routed port.

SVI

With this method, I would create a VLAN 10 on my switch for subnet 192.168.10.0/24. I would assign an IP address from that subnet to my router's LAN interface and connect it to an access port that is part of VLAN 10 on the switch.

Routed port

A routed port is a port that behaves like a port on a router. It handles layer 3 protocols. Using this method, you need to create a small network between your router and the switch. You can set the port to become a router port by using the "no switchport" command when configuring your interface. Then, you assign an IP address to that interface. This small network only has two members in it, so it is a /30 subnet. I used 192.168.254/30 and used two IPs in there. So basically, with a routed port, we are creating a peer-to-peer network that will bridge the networks known by the router and the ones known by the router. On the switch, I would configure the routed port like this:

interface fast 0/1 no switchport ip address 192.168.254.2 255.255.255.252

Either way, I need to create routes on the router because the router will not be aware of the other subnets on my switch. It will only be aware of the peer-to-peer network (if using a routed port) or the subnet for the vlan it is part of (if using SVI). But it will not know anything about the other subnets in other vlans. At first, I thought about assigning the router's LAN port an IP address in the VLAN1 subnet and connect it to an access port for VLAN1 on the switch. But then, since I have an ACL that prevents VLAN2 from talking to VLAN1, the VLAN2 wouldn't have access to the router, so no internet for VLAN2. So I thought about creating a VLAN 10 in which my router would be part of, in a new subnet 192.168.254.0/30. Only the SVI on the switch and the physical port on the router would belong to that subnet. But why do this when a routed port does exactly that anyway? So I chose to go with the routed port even though it does the same thing as the SVI method

Routes

The router only know about network 192.168.254.0/30. But because we are doing TCP/UDP port forwarding, it needs to know about 192.168.1.0, 192.168.2.0, 192.168.3.0 etc... So I tell the router: Whatever you have for 192.168.0.0/16, throw it on the FastEthernet0/0 interface. Whatever else you got, ship it on the WAN (my DSL modem). This is done like this:

ip route 192.168.0.0 255.255.0.0 Fast0/0 ip route 0.0.0.0 0.0.0.0 Dialer1

For the switch, things are really simple. The switch knows how to route between VLANs because we have created an SVI for each VLAN. We just need to tell it to send whatever does not belong to a known subnet to the router, through the routed port:

ip route 0.0.0.0 0.0.0.0 Fast0/1

ACD SERVER WITH RESIPROCATE

2012-07-09

This is another project I've been working on using resiprocate. I'm using this ACD server with Asterisk. All devices, including the ACD server are talking with asterisk. The ACD server registers with asterisk just like any other phone would. When my gateway (FXO) sends a call to asterisk, asterisk routes it to the ACD server. When an agent logs on the ACD, the ACD server checks the contact header and establishes a presence subscription to that contact. In my case, this is always going to be an extension on Asterisk.

Features

Feature list:

  • multiple queues
  • Agents can be members of many queues
  • Queues are called with AOR (i.e: telemarkerqueue@acdserver.local)
  • most-idle agent routing (MIA)
  • redirect on no answer (to next MIA, with configurable timeout)
  • music while waiting
  • welcome message + periodic announcements while in queue
  • supports g.711 uLaw only
  • supports SIP info only for DTMF
  • REST api with JSON formatted responses
    • list of queues and calls with state, source/destination etc.
    • list of agents with state, idle time etc.

Unfortunately, there is too much I want to do so here is the list of other features I would like to add when I find the time

  • Adding agents & queues dynamically without restart
  • Ringall queues
  • Calling an agent directly (agent@acdserver.local)
  • Force a call out of the queue using REST api (unattended transfer to any device on same call server).
  • Use one thread only for all RTP sessions
  • Prevent agent to log on more than one phone and more than one agent on same phone whatever codec I use)

Using it

Here is a typical scenario:

  • Agent dials *44 (asterisk routes this to sip:acdlogin@acdserver.local as per dialplan)
  • ACD Server answers and prompts for agentID
  • ACD finds agent in internal list
  • ACD subscribe with calling phone (using contact header).
  • agent becomes available when a notify indicates IDLE.
  • agent dials *45 (asterisk routes this to sip:acdlogout@acdserver.local as per dialplan)
  • ACD unsubscribes and set Agent as unavailable

Asterisk

In sip.conf, under the profile for my acdserver, I set:

subscribecontext=acdpresence context=acdserver

When phone1 calls the acdserver, the contact header will appear to be sip:phone1@asterisk.local to the ACD server (because asterisk is a B2BUA). So when the ACD server will try to subscribe to that device, it will need to have a corresponding entry in the dialplan. This is how I setup my dialplan:

[acdserver] exten => _.,1,Dial(SIP/${EXTEN}) [acdpresence] exten => _.,hint,SIP/${EXTEN}

It is discouraged to use "_." but currently, this is my only option. I'll try to find something. But this is because I don't have a naming convention for my devices. If all your devices are called phone1, phone2, phone3 etc. then you could use exten => _phone.,1,Dial(SIP/${EXTEN})

API

The server listens for incomming requests in the form of a RESTful API. The responses are sent as JSON data. I prefer JSON over xml since it is easier to parse with javascript and it also looks nicer in my opinion. I used my own json library which you can also find on this website. The server currently supports 2 requests.
curl -X GET "http://127.0.0.1:242/queues" would return

{ "queues": [ { "name": "queue1", "calls": [ { "id": "335099e931f09ad46ea75b8a451ad65d@192.168.0.3", "state": "assigned", "from": "gateway@192.168.0.3", "to": "queue1@192.168.0.3:5071" } ] }, { "name": "queue2", "calls": [ ] }, { "name": "queue3", "calls": [ ] } ] }

curl -X GET "http://127.0.0.1:242/agents" would return

{ "agents": [ { "id": "2771", "state": "idle", "idletime": "14", "device": "", "memberof": ["queue1","queue2","queue3"] }, { "id": "2772", "state": "loggedout", "idletime": "0", "device": "", "memberof": ["queue2"] }, { "id": "2773", "state": "loggedout", "idletime": "0", "device": "", "memberof": ["queue3"] } ] }

Download

This code is experimental and is a mess right now. A lot of it can change at any time. The only libraries you will need is resiprocate and ortp.
download

C++ JSON LIBRARY

2012-07-09

After spending some time trying to find a good JSON library for C++, I realized that all the libraries out there are too heavy to use. Some of them look very good but their usage looks heavy. So I decided to write my own. My library is compliant with RFC 4627 except that it doesn't support unicode and numbers in exponential format .

Seriously, this library is really easy to use and has no dependencies (other than STL). I cannot find another C++ json library that is that simple to use.

Usage examples

The library exposes one object that is used to do everything you need. the "JSON" object. So there is no need to include a whole bunch of header files and use a whole bunch of class. You only need the JSON object to do everything you need. The object exposes these methods:

MethodDescription
JSON& operator[](int i);Access a list item
JSON& operator[](std::string str);Access an object member
std::string str();Get value of item
void parse(std::string json);Parse a JSON document
std::string stringify(bool formatted=false);serialize JSON object
JSON& addObject(const std::string& name="");Add object
JSON& addList(const std::string& name="");Add List
JSON& addValue(const std::string& val,const std::string& name="");Add string value
JSON& addValue(int val, const std::string& name="");Add integer value
JSON& addValue(double val, const std::string& name="");Add double precision FP value

Reading a JSON document

The library is very simple to use. Just compile it and it will output a "test" executable and a .a that you can link against. Then let's say you have the following JSON document:

{ "obj1":{ "member1":[ "val5", "val4" ], "member2":"val3" }, "list1":[ "listItem1", "listeItem2", { "listObject1":"val2" } ], "value1":"val1" }

The following code is an example on how to use the library:

JSON json; std::string val; std::string str = someFunctionThatReadsAJSONDocumentFromFileOrNetworkOrWhatever(); json.parse(str); val= json["obj1"]["member1"][0].str(); // would give "val5" val= json["list1"][1]["listObject1"].str(); // would give "val2"

Each access to a member will return a JSON object. So you only have 1 class to use at all time. So you can create a new variable each time or you can access all members by chaining the function calls.

val= json["obj1"]["member1"][0].str(); // would give "val5" JSON& j1 = json["obj1"]; JSON& j2 = j1["member1"]; val = j2[0].str(); // would give "val5" val = j2.str(); // would give "{list}" val = j1.str(); // would give "{object}"

Invalid paths

The nice thing about this is that you don't need to worry about null objects. If you try to access an invalid member, you will get an invalid JSON object. But if you try to access another member from an invalid JSON object, you will also get an invalid JSON object. You will never get a NULL object that could crash your application.

val= json["obj1"][1].str(); // would give "{invalid}" because obj1 is an object, not a list val= json["obj2"].str(); // would give "{invalid}" val= json["obj2"]["member2"].str(); // would also give "{invalid}" val= json["list"][100].str(); // would give "{invalid}" val= json["list"][100]["something"].str(); // would also give "{invalid}"

Writing a JSON document

There are 3 methods provided to add items in the JSON document:

  • addObject(const std::string& keyName="")
  • addList(const std::string& keyName="")
  • addValue(const std::string& val, const std::string& keyName="")

All 3 functions have an optional keyName parameter. That is because if you add an item to an Object, you need to specify the key name that will be used in the parent. Again, I wanted to have a simple interface without having to force the programmer to use different classes if using a list or an object. So this here is the behavior of those function calls if you provide the key name or not.

ActionResult
addXXX on Object and provide key name item added and parent uses keyName
addXXX on Object and don't provide key name item added and a key name is auto generated
addXXX on List and provide key name item added and key name ignored
addXXX on List and don't provide key name item added
addXXX on Value and provide key name operation is ignored
addXXX on Value and don'tprovide key name operation is ignored

After adding items in the json object, you can then serialize it with stringing().

JSON json; json.addObject("obj1"); json.addList("list1"); json.addValue("val1"); // key autogenerated because key name not provided json["list1"].addValue("item1"); json["list1"].addValue("item2"); std::string val = json.stringify(true);

Would output:

{ "obj1":{ }, "list1":[ "item1", "item2" ], "key2":"val1" }

Download

Project can be found on github