Sometimes we need to know the country of the user who's using our application for different purposes, such as:
Automatically selecting the most appropriate language
Sending a 302 to redirect the user to the closest POP from his location
Allowing only a single country to browse the site for legal reasons
Blocking some countries we don’t do business with or which are the source of most web attacks
IP Based Location
To achieve this, the most “reliable” information we have from a user is his IP address.
But, it's not as reliable as we could hope for the following reasons:
it’s easy to use a proxy installed in a foreign country to fake your IP address
GeoIP databases are not accurate
GeoIP databases rely on information provided by the ISP
any subnets can be routed from anywhere on earth
When an ISP makes a request for a new subnet to its local RIR, it has to disclose the country where it will be used. This country is supplied as a code consisting of two letters normalized by ISO 3166.
You can use the whois tool to know the country code of an IP address:
whois 1.1.1.1
[...]
country: AU
[...]Geolocation Definition
Geolocation is the process of linking a third party to a geographical location. In simpler words, it's the country of a client's IP address. On the Internet, such a base is called GeoIP.
Geolocation Database
There are a few GeoIP databases available on the Internet, and most of them use IP ranges to link an IP address to its country code. An IP range is simply a couple of IP addresses representing the first and the last IP address of a range.
It might correspond to a real subnet, but in most cases, it doesn’t.
In example:
"1.1.2.0","1.1.63.255","16843264","16859135","CN","China"What’s the Issue with HAProxy, Then?
HAProxy can only use CIDR notation with real subnets. It means we have to turn the IP ranges into CIDR notation. This is not an easy task since you must split the IP range into multiple subnets. Once done, we’ll be able to configure HAProxy to use the subnets in ACLs and do anything an ACL can do.
For example, the range above should be translated to the following subnets:
1.1.2.0/23 "CN"
1.1.4.0/22 "CN"
1.1.8.0/21 "CN"
1.1.16.0/20 "CN"
1.1.32.0/19 "CN"Now, you can understand why GeoIP databases use IP ranges: it takes fewer lines 🙂.
Solution: IPrange Tool
To ease this job, Willy released a tool called iprange in the HAProxy sources contrib directory. You can find it in HAProxy’s git. It can be used to extract CIDR subnets from an IP range.
IPrange Installation
Just download both Makefile and iprange.c then run make:
make
gcc -s -O3 -o iprange iprange.cIprange Usage
Iprange takes a single incoming format composed of 3 columns separated by commas:
first IP
Last IP
Country code
For example:
"1.1.2.0","1.1.63.255","CN"In this example, we’ll work with the MaxMind Country code lite database.
The database looks like this:
$ head GeoIPCountryWhois.csv
"1.0.0.0","1.0.0.255","16777216","16777471","AU","Australia"
"1.0.1.0","1.0.3.255","16777472","16778239","CN","China"
"1.0.4.0","1.0.7.255","16778240","16779263","AU","Australia"
"1.0.8.0","1.0.15.255","16779264","16781311","CN","China"
"1.0.16.0","1.0.31.255","16781312","16785407","JP","Japan"
"1.0.32.0","1.0.63.255","16785408","16793599","CN","China"
"1.0.64.0","1.0.127.255","16793600","16809983","JP","Japan"
"1.0.128.0","1.0.255.255","16809984","16842751","TH","Thailand"
"1.1.0.0","1.1.0.255","16842752","16843007","CN","China"
"1.1.1.0","1.1.1.255","16843008","16843263","AU","Australia"In order to make it compatible with the iprange tool, use cut:
$ cut -d, -f1,2,5 GeoIPCountryWhois.csv | head
"1.0.0.0","1.0.0.255","AU"
"1.0.1.0","1.0.3.255","CN"
"1.0.4.0","1.0.7.255","AU"
"1.0.8.0","1.0.15.255","CN"
"1.0.16.0","1.0.31.255","JP"
"1.0.32.0","1.0.63.255","CN"
"1.0.64.0","1.0.127.255","JP"
"1.0.128.0","1.0.255.255","TH"
"1.1.0.0","1.1.0.255","CN"
"1.1.1.0","1.1.1.255","AU"Now, you can use it with iprange:
$ cut -d, -f1,2,5 GeoIPCountryWhois.csv | head | ./iprange
1.0.0.0/24 "AU"
1.0.1.0/24 "CN"
1.0.2.0/23 "CN"
1.0.4.0/22 "AU"
1.0.8.0/21 "CN"
1.0.16.0/20 "JP"
1.0.32.0/19 "CN"
1.0.64.0/18 "JP"
1.0.128.0/17 "TH"
1.1.0.0/24 "CN"
1.1.1.0/24 "AU"Country Codes & HAProxy ACLs
Now we’re ready to turn IP ranges into subnets associated with a country code. We still need to be able to use it in HAProxy. The easiest way is to write all the subnets concerning a country code in a single file.
$ cut -d, -f1,2,5 GeoIPCountryWhois.csv | ./iprange | sed 's/"//g'
| awk -F' ' '{ print $1 >> $2".subnets" }'And the result is:
$ ls *.subnets
A1.subnets AX.subnets BW.subnets CX.subnets FJ.subnets GR.subnets IR.subnets LA.subnets ML.subnets NF.subnets PR.subnets SI.subnets TK.subnets VE.subnets
A2.subnets AZ.subnets BY.subnets CY.subnets FK.subnets GS.subnets IS.subnets LB.subnets MM.subnets NG.subnets PS.subnets SJ.subnets TL.subnets VG.subnets
AD.subnets BA.subnets BZ.subnets CZ.subnets FM.subnets GT.subnets IT.subnets LC.subnets MN.subnets NI.subnets PT.subnets SK.subnets TM.subnets VI.subnets
AE.subnets BB.subnets CA.subnets DE.subnets FO.subnets GU.subnets JE.subnets LI.subnets MO.subnets NL.subnets PW.subnets SL.subnets TN.subnets VN.subnets
AF.subnets BD.subnets CC.subnets DJ.subnets FR.subnets GW.subnets JM.subnets LK.subnets MP.subnets NO.subnets PY.subnets SM.subnets TO.subnets VU.subnets
AG.subnets BE.subnets CD.subnets DK.subnets GA.subnets GY.subnets JO.subnets LR.subnets MQ.subnets NP.subnets QA.subnets SN.subnets TR.subnets WF.subnets
AI.subnets BF.subnets CF.subnets DM.subnets GB.subnets HK.subnets JP.subnets LS.subnets MR.subnets NR.subnets RE.subnets SO.subnets TT.subnets WS.subnets
AL.subnets BG.subnets CG.subnets DO.subnets GD.subnets HN.subnets KE.subnets LT.subnets MS.subnets NU.subnets RO.subnets SR.subnets TV.subnets YE.subnets
AM.subnets BH.subnets CH.subnets DZ.subnets GE.subnets HR.subnets KG.subnets LU.subnets MT.subnets NZ.subnets RS.subnets ST.subnets TW.subnets YT.subnets
AN.subnets BI.subnets CI.subnets EC.subnets GF.subnets HT.subnets KH.subnets LV.subnets MU.subnets OM.subnets RU.subnets SV.subnets TZ.subnets ZA.subnets
AO.subnets BJ.subnets CK.subnets EE.subnets GG.subnets HU.subnets KI.subnets LY.subnets MV.subnets PA.subnets RW.subnets SY.subnets UA.subnets ZM.subnets
AP.subnets BM.subnets CL.subnets EG.subnets GH.subnets ID.subnets KM.subnets MA.subnets MW.subnets PE.subnets SA.subnets SZ.subnets UG.subnets ZW.subnets
AQ.subnets BN.subnets CM.subnets EH.subnets GI.subnets IE.subnets KN.subnets MC.subnets MX.subnets PF.subnets SB.subnets TC.subnets UM.subnets
AR.subnets BO.subnets CN.subnets ER.subnets GL.subnets IL.subnets KP.subnets MD.subnets MY.subnets PG.subnets SC.subnets TD.subnets US.subnets
AS.subnets BR.subnets CO.subnets ES.subnets GM.subnets IM.subnets KR.subnets ME.subnets MZ.subnets PH.subnets SD.subnets TF.subnets UY.subnets
AT.subnets BS.subnets CR.subnets ET.subnets GN.subnets IN.subnets KW.subnets MG.subnets NA.subnets PK.subnets SE.subnets TG.subnets UZ.subnets
AU.subnets BT.subnets CU.subnets EU.subnets GP.subnets IO.subnets KY.subnets MH.subnets NC.subnets PL.subnets SG.subnets TH.subnets VA.subnets
AW.subnets BV.subnets CV.subnets FI.subnets GQ.subnets IQ.subnets KZ.subnets MK.subnets NE.subnets PM.subnets SH.subnets TJ.subnets VC.subnetsWhich makes subnets available for 246 countries!
In this example, the subnets associated with Australia are:
$ cat AU.subnets
1.0.0.0/24
1.0.4.0/22
1.1.1.0/24
[...]The bash loop below prepares the ACLs in a file called haproxy.cfg:
$ for f in `ls *.subnets` ; do echo $f |
awk -F'.' '{ print "acl "$1" src -f "$0 >> "haproxy.cfg" }' ; done$ head haproxy.cfg
acl src A1 -f A1.subnets
acl src A2 -f A2.subnets
acl src AD -f AD.subnets
acl src AE -f AE.subnets
acl src AF -f AF.subnets
acl src AG -f AG.subnets
acl src AI -f AI.subnets
acl src AL -f AL.subnets
acl src AM -f AM.subnets
acl src AN -f AN.subnetsContinent Codes & HAProxy ACLs
Fortunately, we can summarize it into continents. Copy and paste the country code and continent relation from the MaxMind website: http://www.maxmind.com/app/country_continent into a file. The script below will create files named after the continent name and the country codes it relates to:
$ for c in `fgrep -v '-' country_continents.txt | sort -t',' -k 2` ;
do echo $c | awk -F',' '{ print $1 >> $2".continent" }' ; doneWe now have seven new files:
$ ls *.continent
AF.continent AN.continent AS.continent EU.continent
NA.continent OC.continent SA.continentLet’s have a look at countries in South America:
$ cat SA.continent
AR
BO
BR
CL
CO
EC
FK
GF
GY
PE
PY
SR
UY
VELet’s aggregate the subnets for each country on a continent into a single file:
$ for f in `ls *.continent` ; do for c in $(cat $f) ;
do cat ${c}.subnets >> ${f%%.*}.subnets ; done ; doneNow we can generate the HAProxy configuration file to use them:
$ for c in AF AN AS EU NA OC SA ; do
echo acl $c src -f $c.subnets >> "haproxy.conf" ; done
