Voice recognition / Speech recognition / Speech to text is with the help of could services such as
Azue
,
Bluemix
,
Google Cloud
, etc. is gradually becoming a realiable way to get user input. And language support is also improving including support to understand different dialects like the Swedish dialect from Skåne (Scania), that dialect is even hard for some swedes to understand. I created a simple use case using our the Voice Call API. And the result was pretty good despite the limited sound quality of the Cellphone network.
Call flow
Here is an overview of how the systems are connected.
Call arrives from someone.
A
welcome-sound
file is played to the caller.
A recording is then started.
When 3 seconds of silence is detected, or when a digit is pressed on the phone, the recording stops.
Once the recording is complete a request is sent to the set endpoint informing it that the recording is now available for download. At this point it’s possible to start the transcription of the recording.
In parallel with the transcription, it is possible to make actions in the call, e.g. playing a sound saying - “wait a second”.
The next step is to check the text content and take an appropriate action.
In this example I connect the call to the correct number depending on the name in the recording.
Endpoints
To handle this I created two endpoints.
/recording
This is the endpoint that will be informed when new sound files are available for a
download in the 46elks API in order to send it to the speech recognition API.
/whatnow
This it the endpoint that will be informed when the caller is ready to be forwarded to the telephone requested. This endpoint needs to wait for the recording to be transcribed.
Soundfiles:
I created 4 sound files:
welcome.wav : "Welcome how can i help you?"
ok_wait.wav : "Ok I’ll see what I can do."
nosound.wav : "Sorry I did not hear what you said, can you please repeat?"
bussy.wav : "The telephone was busy, what do you want me to do?"
Parts on of the process
First step is to add a start JSON to voice_start on the number for the incoming call.
Initial voice start
The call starts with the welcome.wav, after that the sound is recorded and then ok_wait.wav is played and lastly the request to the
/whatnow
endpoint is made.
When the BASE64 string is available the request to the Google Speech API is made. The reason for using the Google Speech API in this example is that is supports the format of the sound files as is received from the 46elks API. And also Swedish along with lots of other languages is supported.
The API will then reply with different options of what the sound could mean as text. A simple solution is to select one with the highest accuracy. And then store that text in a local file, database, queue, etc.
Wait for recording to be transcribed then handle text.
The transcription may not be fast enough, so some time might be needed for the file to be created. And then take action. In this example I waited for 13 seconds before concluding that the transaction failed.
In 2016 46elks decided to implement
Mondido
as a fallback payment gateway.
The idéa was to use Mondido if there were issues with the
primary payment gateway. This was why the integration was started. The reason we selected
Mondido as fallback gateway was:
No fixed fees.
Seemed simple to implement.
We did know about them from the SUP46 community.
However after running our primary gateway for a while we did notice some horrible problems:
They were unable to prevent the majority of payments with stolen cards and
we had to cover the cost of sent messages with stolen credits by refunding the payments.
The payment reports for the book-keeping where very hard to handle resulting in lots
of work for the book-keeper.
The primary gateway needed a complex SDK to work in a simple way.
During an error with an update of the SDK, we did an emergency switch to Mondido.
Since the switch we have been using Mondido, and we are very happy with that.
3DS
The 3D Secure (
Verified by Visa
,
MastercardⓇSecureCode™
,
American Express SafekeyⓇ
, and
JCB J/Secure™
) provides the safety we need, since the number of stolen cards is reduced
close to zero, and even when there is a stolen card we do not have to cover the cost.
For us stolen cards are extra problematic since the messages sent are often
phishing SMS or spam. This could have caused the operators to block our connection to
the telecom network in the long run.
Real contract
With the original gateway we did have one contract that included PSP and card acquirer
in one, this was easy to get but left us with very little control. The acquirer we have
with Mondido does allow us to negotiate the contract for example the number of days we
have to wait for the money. And the acquirer (
Teller
) is huge and even if we had to wait
for the payment for twenty days, (in reality we wait for as short as two days) the
economic risk is close to zero. Teller also gives simple reports for book keeping.
This saves us at least 4 hours of bookkeeping monthly.
Best entry level price
When we decided to check the price details for Mondido we noticed that the price was
extremely appealing. Even better than our primary gateway. Mindido does give different
price levels for expensive cards like AMEX (1,9%-3%) and low cost for cards like
Visa/Mastercard (
1,89%
), instead of giving a high start cost percentage for all cards.
Great API
When implementing the
API
some other great benefits were found:
Secure
The payment system adds extra safety for us because:
Only our Backend can initiate a payment process by issuing a payment order-ID. This
way even payments failing anywhere in the process can be tracked and completed. This
way we can follow failed attempts thus identify and block fraudulent accounts.
We never store the card details not even a token. The data is sent directly from the
page to the Mondido backend. Even for the stored cards we can just query Mondido for the
token when needed, all we need to do is keep a secure API password.
SSL with Extended Validation is used by Mondido, this can be used by payment window
to ensure the user that the card details are handled by Mondido, if needed. Or you may
even have a sales page with http (I do not recommend it but you could) or have a page with
Let’s Encrypt
for the page and hosted window for the payments.
Simple
It was simple to implement the API, there were
SDK
s available but using our own
framework was equally simple. The API is closely integrated with our system and
therefore it is a low amount for code and in turn less bugs. The endpoints are easy to
understand. However one thing that was hard to understand was how to use the the filter
functionality when querying data from the API.
Costumer with reference
We do supply our user id to the payment, this makes it easy to follow payments for a
single user as well as a simple way to verify payments. When we created the integration
we did think that this was a problem because adding the customer ID required 10-20 lines
of extra code. But we are very happy that we did implement it because the benefits are
worth it! It is possible to skip this by simply using the same ID all the time,
but don´t, you would probably regret it.
One line code for storage of card
When we wanted to add recurring payments we spent quite a few hours planning how
to implement it with the old payment gateway, and it became so cumbersome that we never
really completed this task. With Mondido it was just one line of code:
<input class="storecard" type="checkbox" name="store_card" value="true">Store my card for future payments.
Resulting in this in the 46elks Dashboard:
The card is stored on the user ID and can be accessed and used in the
API.
JS or Hosted page
One benefit we have not completely taken an advantage of is that there is a JS and a
hosted page option for payment, this means that we could offer SMS payment for example
by simply sending a link to a hosted payment in Mondido. We are only using the JS version
this is great for giving the user the on page experience. But what is great with the
in hosted page is that you can change everything on it. We have not used these options
to a huge extent, but I'm sure there is lots of innovation to be done with these
options.
More EU options available
Mondido is based in EU (Sweden) and understands that not all is about Credit Cards, so
they also offer invoice payment, etc. They also support local payment methods like Swish. This
is very interesting for us since it lowers the barrier to get started with our platform.
International support
The payments from countries around the world work great as well. We were not sure that
this would be the case. But our worries were proven wrong and international credit
cards have been working very well.
They do not market well for developers
For us the major benefits of using Mondido do not really match their primary marketing slogans, which are:
We help you convert more visitors into paying customers.
Increase Sales and lower your costs with Mondido.
Mondido offers card fees of 1,89 SEK + 1.89% per payment.
A/B testing support.
These are not very interesting things for a developer. I do think the major benefits with Mondido from our experience have been these:
Mondido allows you to adapt your user experience in every detail.
Mondido is simple to implement, keeping the integration cost low.
Mondido does not have a “one size fits all” contract but a base one that is adapted
to the customer needs along the way.
I work at a company called 46elks, one part of my job is to make events that give us the
opportunity to show developers that we exist. I learned about 46elks in 2013
during an event called Travelhack, organised by
Trafiklab
. In the same event I also got to
know the company Oxyfi.
The idea
In december 2014 I joined the 46elks, and in the spring 2015 we started to discuss
good Hackathon themes. An idea to have a hackathon on a train was voiced by Andreas Kron.
I contacted Oxyfi, who luckily were equally excited about the idea! I also talked to the train companies and
Blå Tåget
was able to help us. Sadly in 2015 there was not enough time to do the
Trainhack
.
The date was set
In the beginning of 2016 we felt that the time was right to push for a
Trainhack
2016!
I started contacted the
Trafiklab
,
Oxyfi
and
Blå Tåget
. The date was set to 2016-08-25
and the exact time (initially) was 08:55 because this was a date when the special conference wagon of
Blå Tåget
was available at a price that was safe enough even in case the event did not attract anyone.
The event is taking shape
The decision to make the event a public transit event was easy. We would depart from Gothenburg
and go all the way to Uppsala to have more time on the train and also be
able to use the 46elks office space upon arrival, in case we did not find a more suitable space.
The food arrangement would be Breakfast and fika on the train and lunch upon arrival (once already in Uppsala)
Public launch
We decided to go public with the event before the summer, and the same day Dag König from
Microsoft signed up and asked if they could be a part of the
Trainhack
. Some more
private signups followed, diverse in ages and backgrounds. Some weeks later I found
out about a company called eTraveli that was into community events as well. So I decided
to ask them if they were interested in sending a team to the event. They responded
with overwhelming support, they offered not only to send their team, but also organize the Uppsala end of the event. Everything felt perfect!
The surprise
The next day, we received information that
Blå Tåget
lost their permission to run trains.
This was not encouraging at all, I halted the work of pushing the event on different channels. And
started to get information from
Blå Tåget
, who promised that they would try to get the
permission back. And luckily some days later they did it, great! However the timetable was taken from
them and some of the departure times were given to another operator. The process of
getting the times back was long but after some weeks the times were secured again. I
started to push for the event again. Read more in
GP
and
Di
.
The 2nd surprise
Two weeks before the event the court took the permission from
Blå Tåget
again! And now
We were really short on time And again great people at
Blå Tåget
did get another operator
Tågab
to agree to
to do the job instead. Now the only thing left was transfering the time from
Blå Tåget
to
Tågab
with no virtually time to spare! The application was sent to the train traffic control
officer. Read more in
GP
and
Di
Great news for the open transit APIs
After all the setbacks the luck was on our side again and the ride started to seem less and less bumpy . The first good news was that
Oxyfi
did
manage to convince the train companies
Norrtåg
and
Värmlandstrafik
to share their GPS data
with a Push API. It’s super useful when it comes to improving the delayed information for
the drivers and the travelers. The second good news was that
eTraveli
gave access to a
price API for air travel so it would be possible for the developers to make an Air+Rail
price-comparison solution.
The 3rd surprize
After 4 days -2 days before the event- I got an email from
Blå Tåget
,
that we did still not have any timetable! There was an option to wait for 2 more days,
and hope for a timetable. If that would fail we would still have to pay for the
train without being able to go anywhere. This was two days before the event so hotel and
train tickets for traveling the other direction were already booked! I was in despair,
and did all I could to get a timetable, but without success.
The flexible train companies
After talking to
Blå Tåget
, a possible solution was found, however, it would require
a maneuver not commonly used in Sweden today. The solution was that the wagons would be
connected to a normal
Tågab
train that would depart from Gothenburg at 06:15 (instead of
08:55 as planned in the beginning). And at the station of Kristinehamn, the wagons
would be disconnected, left in the station and the other part of train would continue
without us. About 40 minutes later another train would have a 7 minute long stop at the station.
During this time our wagons would be connected to that train and it would depart
to Stockholm.
Blå Tåget
and
Tågab
worked together and were able to make this
possible! And there was hope again!
Replanning
The new travel time resulted in new planning for the food on the train. But more importantly,
it required that everyone attending the event needed to replan their mornings! So I stated
sending information to all the participants about the new time. As an additional issue
the pre-event place also cancelled, so I would also have to find the pre-event location, for the meetup, alongside the
food and beverages. Lots and lots of things to plan. How were we going to arrange the food
for the train? Nice people at the
Creative Loop
in Gothenburg provided a solution for the
Pre-event space and supported us with how we could solve the issue with the food. As it
turned out, the train would be standing on the platform during the night so we could leave the food on the train in the evening. So now we did have a plan that could work.
Creative Loop
also helped with sleeping arrangements for the Photographer from Nerdy Makers.
The heart attack
The day before the event,
Blå Tåget
calls me and says "We have a problem...".
My heart rate jumped to 220 bpm and blood pressure to 230/180! No, I can't take one more setback!
The message in the phonecall continued "...but we were able to solve it quickly. ".
So my body dropped the heart rate and blood pressure. However the adrenaline in my body would stay at
levels I never have felt before.
The missing prices
Due to all the issues with the train and the timetable, some other parts of the planning had
lagged behind. One quite important part was to arrange prices for the winners of the event, so the day before the pre-event we only had one price and that was a ticket and boat trip to
Junction hack
in Helsinki (in November). I decided to buy several
Raspberry Pi3s
to have them as prices. I remembered that I knew someone from
Webhallen
so I decided to call him. As a result, Webhallen sponsored the event with some Raspberry Pis! This was great! Unfortunately, I was already on my way to Gothenburg but my colleague, who was sick and therefore stayed in Uppsala, could pick up the Raspberry Pis. Also,
Trafiklab
promised to arrange one more price for the team that would impress them the most.
The water boys
The water on the train cannot be used to make coffee because the quality cannot be
secured, so all water needed for drinking and coffee had to be brought along by us. So me
and Rob took all the empty PET bottles from my apartment in Gothenburg, filled them up and
put them in an IKEA bag to carry them to the train in the evening.
The Pre-event
Then it was time to take care of the pre-event. Food needed to be arranged, thus, a quick run to the local supermarket for cheese pie was the solution. Later on, the travelers arrived and started to socialize. When the night was getting closer it was time to go and buy the food for
the train. Everyone from the event decided to help so it turned out to be very easy to arrange.
The food odyssey
The
Trainhack
ers participating in the pre-event went together to the train station in Gothenburg, bringing the water bottles. There we met Joel, the train manager, who was going to help us
along the way. We then went to Hemköp and started to buy things from the long shopping
list. With the help of Matilda, from Microsoft, and Rob, from Nerdy Makers. The rest of the
people stayed outside of the store and waited for us to return. After a while we were
interrupted by a security officer. Apparently, the shop had been closed for 10 minutes already.
So once more some adrenaline and pulse to finish shopping. When we came out of the
closed store, the company had left because the guards said that the store was closed.
As a consequence, we needed to carry the heavy shopping bags all the way to the train,
where luckily Joel was ready to help us put the food in the refrigerators.
The late night planning.
After the food odyssey, me and Rob went back to
Creative Loop
, where we met Victoria
also from 46elks. She was already working on the
Trainhack
logos to put on
the doors and on the walls to make the event more visible. We also created the diplomas for
the winners. After some hours we were finished and went to bed.
D-Day
I woke up early in the morning, no alarm needed, still lots of adrenaline in the body
apparently. I took the remaining things and walked to the station. At the station I did
see the train still there. And to my surprise the hackers started to arrive, and they all
seemed happy and ready to go. Some minutes later Joel arrived and opened the wagons for us.
Oxyfi arrived to support with the on board wifi. The day before all the antennas did not
fit on top of the conference wagon so
Blå Tåget
added one extra wagon for us, just
like that! So we did get a 45 ton 3/4G antenna on the hock. The extra wagon was also quite good for comfort since traveling sideways for hours is not for everybody.
Departure
When the train departed from Gothenburg station, I could not believe it! I was in shock. My
colleague Victoria reminded me to welcome everyone. At that point I understood that
Trainhack
2016 was happening for real. During this time, Jakob from
eTraveli
, Viktoria
from
Oxyfi
and Matilda from
Microsoft
, as well as Victoria from 46elks where arranging the
breakfast buffet. With the buffet, the first
Trainhack
in the nordics was started. Victoria
stated the group presentation where everyone presented themselves and what they would
like to do.
The stolen train
The onboard atmosphere quickly turned to relaxed. I felt that everyone
knew each other, or was at least eager to do so. Some minutes into the trip the train was delayed some minutes due to a
slower train ahead. To regain the lost minutes the driver did the best he could to regain
the time, i.e. full acceleration and full braking and max allowed speed all the time.
So basically driving like he stole the train, so we all sat tight and looked for things to hold on everytime the train would start to slow down. This gave some more experience and motion sickness to some of the hackers.
The Wifi starts to fail
At some point our fast internet connection started to malfunction. Luckily
Oxyfi
was on the train and
started to work on figuring out what was wrong. The issue was first believed to be due to one of the new antennas.
But later on it seemed like the issue was more complicated, with TCP errors that prevented new sockets to be created.
The issues were never really solved completely during the trip. However,
Oxyfi
got lots of
information of how the system could be improved in future. Especially when it comes to lots of
short SSH sessions to Github, at the same time as there is live streaming going on from the same network.
So for next year the connection will be greatly improved. Later on they found errors that does only apear when
the train is running att speed. The result of this is that internet onboard from
Oxyfi
for current trains
and the future trains will become more stable.
SJ helps out
When the train was getting closer to Stockholm, I did remember we needed to get all the
way to Uppsala. I needed to buy 24 tickets from Stockholm to Uppsala. However, that was
not too easy to do on the
SJ
webpage. Luckily we did have a
SJ
developer on the train
and he helped me with buying 24 tickets both quickly and easy. He also managed to get Lounge
access for everyone while we were waiting for the train to Uppsala.
The countdown
After arriving to the
eTraveli
office in Uppsala, we were welcomed by Jakob Rödström who
presented the place and informed everyone that the food was served. All
Trainhack
ers started eating and kept on hacking. It got pretty quiet while everyone was trying to finish their projects. The time flew by, and at 16:30 the presentations began.
For next year I hope we can have much fewer surprises. There have been some great things happening after the event,
SJ
showed an interest in being a co-organizer and all the organicers from this year have also
been very eager to keep cooperating which makes me hope that everyone will join us next year as well.
For the future prehaps a cooperation with other Trainhacks like
Hacktrain.nl
could become a possibility.
Next year the rout might be different perhaps Mamö - Stockholm - Gothenburg could be an interesting route.
If you are interested you can signup for information for the next year at
Trainhack
.
To play sound files to the users with the 46elks voice API you need to have somewhere to store your sound files. One solution is to store the files in your Dropbox and use the download link in the API.
Add a file to your dropbox folder
Open the folder in your Dropbox (where you want to store the sound file). Upload the sound file by selecting upload:
Share the soundfile
Share the soundfile by clicking on
Share
:
Create link
In the popup select
Create link
:
Copy the link
Then copy the link:
Change the link for direct download:
Change from:
https://www.dropbox.com/s/gi4dfokh0srnpij/hello.wav?dl=0
to
https://dl.dropbox.com/s/gi4dfokh0srnpij/hello.wav
by changing
www
to
dl
and removeing
?dl=0
You may allso change the:
https://www.dropbox.com/s/gi4dfokh0srnpij/hello.wav?dl=0
to
https://www.dropbox.com/s/gi4dfokh0srnpij/hello.wav?dl=1
by changing
?dl=0
to
?dl=1
Update your JSON in the dashboard:
The next step is to save the url in the JSON for the number:
You may use the file in any JSON for the Voice calls with 46elks.
I feel concerned with the hackathon climate in Sweden right now. I keep hearing comments about Hackathons being flawed, while I don't think hackathons are perfect the reasons that are often mentioned to highlight the imperfections are in my opinion quite inaccurate.
The flaws that are often mentioned are:
There are few products that result from Hackathons and most projects in or after Hackathons are not commercialised.
Inexistant problems are solved. Teams are creating solutions that are not needed in real life.
After the hackathon the groups do not stay together (meaning that most of the time no company is created as a result of a co-operation at an event).
I don't share this point of view. From my perspective these are not issues with Hackathons. For me a Hackathon is something different.
I believe that Hackathons are great because:
They serve as a source of inspiration, your own teammates and other teams involved in a Hackathon can be a great source of inspiration.
They can be a place where developers with different backgrounds can attempt to bring their ideas to life without worrying about the feasibility of a possible final solution, or the need of having a market segment (economic support etc.).
They offer a unique feeling that everything is possible, when the event starts and one sets out for the unknown.
They are normally a fun event to enjoy, it is not work! Social networking with new friends or a fun project with old ones (and possibly even enemies).
They offer a learning experience on how to solve an issue you did not know needed solving in the first place, which I think is a great way to learn new stuff.
They do not require long term commitment. Entering a Hackathon is a small commitment, no need to maintain any product beyond the event.
They do not require signing of contracts or other the other kind of hustle that is not development.
They give the opportunity to take part in a competition with others, that may push you to do a little extra.
Hackathons and Solutions
A Hackathon does not always need to arrive at a final solution. Often it is enough to raise attention around a certain problem and develop plausible solutions that can be taken to the next level by the society.
Hackathon is a marathon of hacking, no one would ask a sports-star to solve an issue with low milk price in the framing business in Sweden. But they are often expected to inspire and be a role model for future generations. I believe that hackathons should be an inspiration for others rather than the final solution.
In my experience issues often are not technical, usually the problem is in our minds, here are some examples:
The trains in Sweden had a top speed of 90 Km/h early on but were limited to 40 km/h because politicians believed that the speed would be harmful.
Internet was not promoted by Telia (Swedish state telecom company) in the beginning because they did not believe it was the future, and created their own product instead. Tele2 did and was a part of swipnet, the beginning of IP networks in Sweden.
When 46elks was created it was extremely hard to have SMS, Voice & MMS over the internet sent to any application. And this was not mainly due to technical reasons - it took years to get the agreements with the operators but just months to get the service running.
My conclusion is that the issues often only require relatively small amount of work from developers. Instead the big part is often the bureaucracy in trying to explain to the general society what needs to be done in a given situation.
What could Hackathon 3.0 be?
Vinova
will announce development funds for improving the Hackations in Sweden. Vinova is an agency that is created to support innovations is Sweden.
I'm also a teacher by profession and all I have learnt, when it comes to how the mind works, tells me that it is super important that hackathons are kept as an open event and are not turned into a forced way of creating a company.
I believe that Hackathons would have the best role in innovations by supporting the community through networking and providing a commonplace for learning new technology. And most importantly show how we can improve the environment we live in without slow bureaucratic systems limiting us
Sending out SMS links is very common, and it is often interesting to know who clicked the link in the SMS. Since the web browser does not know what phone it is running on - a unique ID is needed for each SMS. It is possible to use an URL shortner like (goo.gl) that makes links like this
http://goo.gl/s6zIe2
.
URL-Shortener limitations
However using URL-Shorteners has some limitations:
Privacy is limited since lots of free solutions publish your links.
Speed is limited since your need to make an API request for each phone number.
Number of possible urls are limited (due to limits in the shortner service).
Cost to avoid the problems of Privacy, Speed and amount limits you may pay for services, however this is often quite expensive.
Encode number as short string
To solve this issues and still keep the number information - you may encode the number as an encoded string. This way a number like 723175800 can be coded as "SWcuk" and put in an example url like:
http://sortu.rl/SWcuk
"SWcuk" can then be decoded to the original number with a simple self contained function. No access to a database is needed.
Benefits
No limit in number of urls created.
Central database for translation urls is not needed.
Low cost.
Number link connection is keep all the way in the flow.
Downsides
Only works for numbers.
Not possible to use for general URLs.
Example of implementation in python
alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz_-~=+/.!&"
charscount = len(alphabet)
def encode(n):
n = int(n)
encoded = ''
while n > 0:
n, r = divmod(n, charscount)
encoded = alphabet[r] + encoded
return encoded
def decode(s):
decoded = 0
while len(s) > 0:
decoded = decoded * charscount + alphabet.find(s[0])
s = s[1:]
return str(decoded)
46elks is now public on
Zapier
the beta period is history. Zapier has also added some nice features like Delays and multiple actions on one trigger.
Great for holidays
I wanted to have a holiday some weeks ago, and i did not want to check my email during that time. However I still wanted to get an update every Friday by SMS regarding the subjects of the emails sent to me exclusively.
I connected the Zapier to my Gmail and connected it to a spreadsheet to have an overview of the messages that were relevant, there I made a filter to avoid getting emails where I was CC’ed or one for several recipients.
The next step was to add the great function of Delay, (a new Zapier native function). That allows you to select in my case "next friday at 15:00".
This is what the flow looks like:
My experience
I’ve learned that it is important to remember to charge the 46elks account with credit before going for a vacation.
I was surprised, how noticeable it was not to have an e-mail notification most of the time. 😎
We at
46elks
are supporting
Kodcentrum
with an application. They use Google Drive and Google Apps Scripts.
I created an endpoint that would be called by the Google Apps' Script every day. Everything was working fine when I was running the script manually, but when I tried to run it with a time-trigger it did not work.
I saw that the trigger did start the script by checking the script-log. But no request was made to the endpoint. Google reported that the service was unavailable whitout even trying to make a request.
I made sure that IPv4 & IPv6 was not the reason. After an extensive research I found the issue. Apparently Google service is using different systems to run manual and time-triggered tasks.
Finally I found that when saving the url to the endpoint in the script, Google server tries to read
/robots.txt
from the endpoint server. The endpoint server I created responded with
50X
for that url. That caused Google to think the endpoint was down completely! And it would not try to make any request to any other path on the server.
To solve the issue I modified the server of the endpoint so that it would respond with 404
to all paths except the correct one, and updating the url in the Google Apps' Script solved the issue!
Do you need to secure the traffic between two nodes on a network you don´t trust? There are many VPN solutions as well as SSL and SSH. They all require settings for either the application or for the link itself to work.
IPSEC encrypts on Level 3 (in the IP stack) because of this the application does not need to know about the encryption.
There are two types of IPSEC modes, Tunnel and Transport. I use Tunnel mode in many cases for standard VPN. Transport mode is not used as often since it has more limited usecases.
I needed to secure traffic between two servers. The servers were initaly on a trusted network but needed to be moved to a network that I do not have full controll of. In my case the data sent over the link is occasionaly sensitive, so I needed a way to make it harder to listen to the link.
The nodes have direct IP access to each other. This example is with internal IPs but it works the same way with public IPs on both machines.
This guide will show you how to setup a static link, wich has a limited amount of packets that can bet sent. The maximum number of packets that can be sent over the link without a reset is 2^32. But if the kernels support 64 bit index the max amount of packets is 2^64.
This is how i did it
Install ipsec-tools
I started by installing ipsec-tools on the two machines.
apt-get install ipsec-tools
Flush IPSEC databases
I changed
ipsec-tools.conf
to flush the ipsec databases when running the setup:
# Flush the SAD and SPD
flush;
spdflush;
Setup
I created a conf file:
/etc/ipsec-tools.d/transport-link1.conf
on the host with ip 192.168.0.2. Allso make sure that the data in this file is secret since it contains the crypto-keys.
#!/usr/sbin/setkey
# for host 192.168.0.2.
# Put the ESP SAs in the SAD, 192 bit long keys
add 192.168.0.3 192.168.0.2 esp 31031 -E rijndael-cbc
0xf86f440c0247a935c75be785486e535a90a636f0b189d588;
add 192.168.0.2 192.168.0.3 esp 43241 -E rijndael-cbc
0xe9749d02219238454dd0dba8eae55155131c86c05a2f7d4e;
# Put the IPs in the Security Policy Database
# to require the traffic to be encrypted
spdadd 192.168.0.2 192.168.0.3 any -P out ipsec
esp/transport//require;
spdadd 192.168.0.3 192.168.0.2 any -P in ipsec
esp/transport//require;
I created a conf file:
/etc/ipsec-tools.d/transport-link1.conf
on the host with ip 192.168.0.3 as well.
Note that the only difference between the files is that the IPs in the "spdadd" lines are swapped
#!/usr/sbin/setkey
# for host 192.168.0.3
# Put the ESP SAs in the SAD, 192 bit long keys
add 192.168.0.3 192.168.0.2 esp 31031 -E rijndael-cbc
0xf86f440c0247a935c75be785486e535a90a636f0b189d588;
add 192.168.0.2 192.168.0.3 esp 43241 -E rijndael-cbc
0xe9749d02219238454dd0dba8eae55155131c86c05a2f7d4e;
# Put the IPs in the Security Policy Database
# to require the traffic to be encrypted
spdadd 192.168.0.3 192.168.0.2 any -P out ipsec
esp/transport//require;
spdadd 192.168.0.2 192.168.0.3 any -P in ipsec
esp/transport//require;
I started the link by the setkey init script on both machines:
/etc/init.d/setkey start
Make sure that the link is working
I tested the link by pinging between the machines and using tcpdump to see the traffic, when the IPSEC is working you get ESP packets when it is not you get ICMP packets in tcp-dump.
Faultfindning
When things go wrong you will notice by the fact that no normal traffic reaches the other host.
I use the tcpdump to listen for esp traffic:
tcpdump -i eth0 esp
To listen to ah traffic, not in the first conf example:
tcpdump -i eth0 ah
There are som basic issues that I have encountered.
The data is sent but not encrypted.
When the data is sent unencrypted it will not be accepted by the other host since
esp/transport//require
requires the traffic to be esp encrypted. The add to the SPD might be the same on the two hosts.
The data is sent encrypted but the receiver does not decrypt it.
The receiver might not be running IPSEC, or does miss the SPD record for incoming traffic.
The receiver/sender have the wrong keys, the keys might be swapped in se SAD. The SAD records should be exactly the same on the two machines. The parameters 31031 & 43241 is called SPI and it is sent in the IPSEC header to tell the the receiver what key to use to decrypt the message, it the SPI is not the same the kernel will not be able find the correct key.
Staring setkey gives incorrect key length error
If you forget the ; at the end of the add statement the setkey gives the error incorrect key lenght. If you forget the 0x before the hex the key is interpreted as ASCI and will therefore also have the wrong lenght.
SAD Security Association Database
The SAD is storing the keys and methods to use for a certain combiantion of IPs. And a SPI in this case 31031, is the ID of the combination of encryptionmethod and key to use.
The SAD is direction aware this means it understands what direction a packet is traveling, and thanks to this the add statement needs to be exactly the same both nodes.
SPD Security Policy Database
The SPD is storing when the kernel should encrypt the traffic. The SPD is not direction aware this means it has to be adopted to the node it is on.
That means that one node needs to say:
spdadd 192.168.0.2 192.168.0.3 any -P in ipsec esp/transport//require;
And the other node needs to say:
spdadd 192.168.0.3 192.168.0.2 any -P in ipsec esp/transport//require;
If it is mixed the data on one node will not be encrypted.
Manpage
I found the
setkey manpage
very helpfull and easy to follow!
Encryption
AES
For the ESP I use the
rijndael-cbc
algorithm also know as AES.
SHA1
When I want AH as well I use the
hmac-sha256
algorithm.
Authentication Header
I did not use Authentication Header(AH), AH does not encrypt the content of the message but it makes sure that the sender is known and that the IP-header is untouched.
Encapsulating Security Payloads (ESP) used in my example, is used for encrypting the data in the packet.
This is a example of AH and ESP in combination:
#!/usr/sbin/setkey
# for host 192.168.0.3
# AH SAs using 160 bit long keys
add 192.168.0.3 192.168.0.2 ah 33362 -A hmac-sha256
0x3308cef208a75c1048182e0a5b11e25a3308ce7a59a1a638ba3245c264538f4a;
add 192.168.0.2 192.168.0.3 ah 30759 -A hmac-sha256
0x3fe6f3d57a58dcedd930de5818ed86be3fe6f33249a2c678bc324f12645f8f4b;
# Put the ESP SAs in the DB, 192 bit long keys
add 192.168.0.3 192.168.0.2 esp 31031 -E rijndael-cbc
0xf86f440c0247a935c75be785486e535a90a636f0b189d588;
add 192.168.0.2 192.168.0.3 esp 43241 -E rijndael-cbc
0xe9749d02219238454dd0dba8eae55155131c86c05a2f7d4e;
# Put the IPs in the Security Policy Database
# to require the traffic to be encrypted
# with esp and ah
spdadd 192.168.0.3 192.168.0.2 any -P out ipsec
esp/transport//require
ah/transport//require;
spdadd 192.168.0.2 192.168.0.3 any -P in ipsec
esp/transport//require
ah/transport//require;
This is a example of AH and ESP in combination for the other host:
#!/usr/sbin/setkey
# for host 192.168.0.2
# AH SAs using 160 bit long keys
add 192.168.0.3 192.168.0.2 ah 33362 -A hmac-sha256
0x3308cef208a75c1048182e0a5b11e25a3308ce7a59a1a638ba3245c264538f4a;
add 192.168.0.2 192.168.0.3 ah 30759 -A hmac-sha256
0x3fe6f3d57a58dcedd930de5818ed86be3fe6f33249a2c678bc324f12645f8f4b;
# Put the ESP SAs in the DB, 192 bit long keys
add 192.168.0.3 192.168.0.2 esp 31031 -E rijndael-cbc
0xf86f440c0247a935c75be785486e535a90a636f0b189d588;
add 192.168.0.2 192.168.0.3 esp 43241 -E rijndael-cbc
0xe9749d02219238454dd0dba8eae55155131c86c05a2f7d4e;
# Put the IPs in the Security Policy Database
# to require the traffic to be encrypted
# with esp and ah
spdadd 192.168.0.2 192.168.0.3 any -P out ipsec
esp/transport//require
ah/transport//require;
spdadd 192.168.0.3 192.168.0.2 any -P in ipsec
esp/transport//require
ah/transport//require;
I'm a typical backend programmer. When I deliver my JSON struct or data stream from the server I think that everything is great. But for some reason a lot of users are not happy with that. I would like everyone to change and accept a JSON struct instead of using a GUI. This does not seems to be happening anytime soon.
Since it seems to take a long time, before I have turned the general public to my way of thinking I will have to make some kind of frontend for people to use. I wanted to make a APP to check the balance of my public transportation card (
SL ACCESS
). I did not have the energy nor the real skills to create a GUI for that. At
Travelhack 2013
I got to know about
46elks
, it was my salvation for this use case.
Instead of creating a complete UI I just connected the backend to incoming and outgoing SMS. The backend loads the account data from the SL webpage to get the balance of each card and sends it to the user. It may look like this:
To make it possible for the backend to access the SL data, the user credentials for SL are needed. This is why the user need to set the credentials as the first SMS to the service. All SMS are sent over the air unencrypted so it is not the best solution. However it is better to send it once instead of sending it every time you need to know the balance.
The next time the user sends anything to the service it responds with the balances of each card. This is very simple and works on any phone anywhere. And hey I did not have to do any GUI work!
However it would be great if
SL
would add support for
OAuth 2.0
for example. To make it possible to avoid sending credentials over SMS. However it would then be required to have access to a web browser on the phone or access to a web browser on a computer.
