Author Archives: geeklawyer

WordPress and Latex

Well, this doesn’t work too well! *sadface*. How can I show you my funky Gaussian distribution?!

$$p(X) = \frac{1}{\sigma \sqrt{2\pi}} \sigma^{-{\frac{(x-\mu)^2}{2 \sigma^2}}}$$

Sooo. While trivial to install Latex plugins on a self-hosted WP instance how is one to do it at WordPress?

Edit: it turns out there is a blindingly obvious ‘plugins’ tab. Huh, obvious if I’d, you know, looked. Catch is that I need to upgrade from free to paid to do this. pffft. What’s a tightwad to do? Easy.

http://www.latex2png.com/

strip off ‘$$’ from the Latex equation, enter in dialog box, insert png file into post and BABOOM:

Advertisement

Machine Learning Masters

I began a Masters degree at Glyndwr university in Wrexham in Data Science & Big Data Analytics. I’m doing it part-time over two years. Why Data Science? It’s an interesting topic; I am very interested in statistics and this is a technological application of that. And while ML is hyped both in its ability to deliver results and its role in industry it is a useful discipline. The course so far is fairly disappointing. I’ve done two modules so far, Advanced Data Structures and Algorithms, and ‘Big data Challenges and Opportunities’. The former is, frankly, just an introductory course in Python and very basic data structures like lists and tree; it would have been more honestly called “introductory”, the latter module is more of a sociology and “ethics of data” course. Both are easy, and not very interesting but great time absorbers. Writing academics papers is a major pain in the arse as the amount of effort to read a bazillion papers then absorb precis and regurgitate them is large and unrewarding. I do hope however that the remainder of the course will perk up and become more technical and engaging.

Edit: the database section of the course was very enjoyable. IT stretched my knowledge of SQL and introduced me to Oracle systems as well as visual analytics which is rather fun.

Finding work when returning to IT

Since leaving behind the Law and choosing to reside in rural England I have struggled more than somewhat in finding a job. Shropshire is a lovely county which I adore and do not intend to leave save by the force of man or fate. However. IT is not a sector that exists in any meaningful way outside of London or a few other pockets of the UK: Brighton Cambridge Bristol being a few. Everywhere else it is dreary webshops and random companies sprinkled about thinly. In my area it is all c# or PHP. To say that I find these languages dull would be like saying “the Greeks have financial problems”.

My own programming love is Erlang but I despite the revelation that it underpinned the success of WhatsApp it is also a London only gig. Most Erlangistas seem to be grizzled of hacks who cut their teeth programming on punch cards. From what I can gather most Erlangers who’ve been only been coding in it for 3 or 4 years are regarded as n00bs. I’ve dabbled on and off for a couple of years but not completed anything: my code so far has just been incomplete fleshing out ideas for projects and learning my way. I’ve not used Dialyzer, packaged up code or used a remote shell to hot-code swap or any of those glamorous things that make Erlang so special. So clearly I’m not ready for a heavy duty Erlang role. Given there are no Erlang junior dev roles anywhere in the UK apparently it seems I can only crack on with hobby projects and release as open sores software on Github if I want credibility.

In the meantime I have to piss about with polishing up my old PHP skills in the hope of landing something tolerable. Yuch,

NO views!

Just looking at my stats & I’m not even viewed by bots. This is good, yes good, news. I can say what I like and it’s like I’m shouting to myself. I can say rude words now: ‘bollocks’. There. I liked that.

Of course failing to write regular posts doesn’t help but that only really applies to blogs with a readership. Then the readership will die off. I have none to begin with. I resolve to write more.

Playing with libevent for PHP

I’ve put a pause on Erlang for a bit to play with PHP in order to write an email server. I stumbled upon Guerillamail’s code on github and thought this might be an example I could start with. Guerillamail is a very well known temporary email address site that I have seen before. It gets a lot of traffic. I assumed their code was war tested. I’d contemplated using Postfix or Exim and driving a Mail Transfer Agent with PHP for my project but the code Guerillamail has on their github site looked battle hardened & able to cope with the C10k problem (when I see a new language I always look at how it copes with this). GM had dropped Exim for their PHP script. My curiosity was piqued and I tried to try to run the code.

Of course Sod’s law meant they’d chosen MySql as their database. To say that I hate MySql would be something of an understatement: it’s always a pig to configure from a fresh install with guaranteed errors that take ages to fix. Unlike PostgreSQL et al it doesn’t Just Work™, and it’s shit by comparison anyway. My solution to this on OSX is just to use MAMP which is a preconfigured set of packages including PHP and MySQL. It generally works well and is a good hack to get a web development environment working: leave it to the devops guys to build a production environment.

What’s nice about Guerillasmtp, and how it solves the C10k problem, is to use Libevent for high performance network socket handling. It all looked good. Too good. I execute the code at the command like thusly:

Guerillamail$ php smtp.php -l log.txt

Guerillamail$

Nada, I just get the command prompt back.

Here is the sorry, unsolved, tale for those Googleing – you are not alone:

MAMP doesn’t have the libevent extension so there’s no libevent.so in /Applications/MAMP/bin/php/php5.6.2/lib/php/extensions/no-debug-non-zts-20131226/. ARRGHH.

Download libevent-0.1.0.tgz and do the Configure Make dance:

phpize; ./configure, make, make install 

no php.h

Oh shit. I need the source code for PHP. The sky darkens & I hear the sound of rolling thunder in the distance; an ominous unseen presence seems to enter the room; the log fire dies inexplicably; in the distance a wolf howls. I have been here before: Configuring Open Sores software.

So it begins. Download & unpack the PHP source code. Configure Make Dance & the source is now in /usr/local/src/php. I read a blogpost that says I can put the source in MAMP by executing:

tar xzvf php-5.6.2.tar.gz -C  /Applications/MAMP/bin/php/php5.6.2/include

Done. Now grab the source to libevent configure & compile:

pecl install libevent-0.1.0

cd libevent-0.1.0

phpize

./configure; make

cp  modules/libevent.so /Applications/MAMP/bin/php/php5.6.2/lib/php/extensions/no-debug-non-zts-20131226/

This all looks good, libevent is in the extension directory so we are good to go! Let’s get this script running:

Guerillamail$ smptd.php -l log.txt

Guerillamail$

 Err, wait, whut? It returned to the CLI prompt? Why did it not just sit there all daemony to serve clients? The log file said nothing. Running the code with the rather useful PHPStorm its console also exits but additionally says ‘exited with code 255‘. Google tells me exit code 255 is an unassigned PHP error code. So that’s as illuminating as a torch with a flat battery. Spiffy. Step though the code then. Great. I got to this bit of code:

283 //log_line("stream base set up");
284 stream_set_blocking($socket, NONBLOCKING);
285 //log_line("stream base - stub0");
286 $base = event_base_new();
287 //log_line("stream base - stub1");
288 $event = event_new();

The script hits line 286 calls event_base_new() and exits (with code 255). The php error log in /Application/MAMP/logs helps somewhat:

[27-Jan-2015 14:47:42 Europe/Berlin] PHP Warning: PHP Startup: libevent: Unable to initialize module
Module compiled with module API=20131226
PHP compiled with module API=20121212
These options need to match
in Unknown on line 0
[27-Jan-2015 14:47:42 Europe/Berlin] PHP Fatal error: Call to undefined function event_base_new() in /Users/david/BTSync/projects/php/PhpstormProjects/whispermailPHP/smtpd.php on line 286

I didn’t realise PHP API calls were versioned. That’s pretty cool I reckon. So API=20131226 and API=20121212 refer to the PHP interpreter API code and the extension API code. I know nothing about compiling PHP extensions but I wonder if there is a mechanism to blow up the extension rather more loudly on an API mismatch like this. Ideally PHP would barf on invocation and before it runs the script, e.g.:

$PHP buggyscript.php

Error module API mismatch: xxxx

Silently failing (stuffing errors in the error log doesn’t count) is less than helpful, especially if code 255 is unspecific. Dunno, just a thought.

By a curious coincidence php5.6.2 has an extensions directory called ‘no-debug-non-zts-20131226’ and that number20131226 looks jolly familiar. I have no idea at this point how the API version number gets in into libevent. At this point I have one of those inspired genius ideas that comes once in a blue-moon: since it’s very unlikely the version number is internally checked against the contents of the module by the module I can load libevent.so in a hex editor and text search for API; and YES, I find ‘API=20131226‘. This I edit it to ‘API20121212’ and reload and run again. Guess what happens!?

Yea, fuck all. OK, so this is going to need more slog less Commando.

Then I just happen to look at the PHP5.5.18 directories an see that the extensions directory is called ‘no-debug-non-zts-20121212‘ WAIT!? 20121212? This is number is the same as the API error. So working hypothesis is that libevent grabs this number and makes it the API version. Fragile. I need to get the source to PHP 5.5.18. Fuck. It can be grabbed from us2.php.net/releases/ in zip form by guessing version numbers. Unpack the zip file convert it to a tar file (erm, yea, I know) into the MAMP directory and rename:

tar zcvf php5.5.18.tar.gz php5.5.18/

tar xzvf php5.5.18.tar.gz -C  /Applications/MAMP/bin/php/php5.5.18/include

mv /Applications/MAMP/bin/php/php5.5.18/include/php5.5.18  /Applications/MAMP/bin/php/php5.5.18/include/php

cd /Applications/MAMP/bin/php/php5.5.18/include/php

So we have the PHP source in the right MAMP directory. Time to Compile libevent. Easy:

[Edit: set the $Path variable or weirdness happens

$set PATH=/Applications/MAMP/bin/php/php5.5.18/include/php/bin:$PATH

]

cd libevent-0.1.0/ libevent-0.1.0

phpize

./configure; make; make install

This leave you with libevent in libevent-0.1.0/ libevent-0.1.0/modules/libevent.so. 

cp  modules/libevent.so /Applications/MAMP/bin/php/php5.5.18/lib/php/extensions/no-debug-non-zts-20121212/

Fire up a new shell bash shell (not sure why some sort of environment change needed?):

Guerillamail$ smptd.php -l log.txt

(waiting)

Yeeesss. I hope, dear random Googler, that that helped.

Choosing an IDE that can cope with several languages

I recently had a yearning to debug using something more sophisticated than print statements. As a technique it works fine (I believe Linus Torvalds uses it in hacking Linux for example) but it does end to leave one’s code splattered with geek droppings: lots of Print “foobar #{var1}” all over the place. In all honestly I feel a bit embarrassed by these when anyone else looks at my code. So I was hoping for a debugger to reduce these droppings. Most languages have a command line debug program like dbg etc but these all have a huge learning curve & are about as friendly a rabid ferret. I hoped to use an IDE to get a well integrated debugging experience where the bells & whistles editing and version control stuff was nice but not a deal breaker.

So what to choose? The three main contenders for OSX Yosemite (I use a Macbook Pro) seemed to be Eclipse, Intellij and Netbeans. There are lots of other IDEs but most seemed to be specific to a language whereas these were all multi-platform and multi-language. I also took the aesthetic choice to avoid Emacs: although it has support for many languages & debug tools it’s a piece of shit. It has a learning curve steeper than the Eiger,  is as usable as a severed 400V cable in the rain, and – visually – it looks like it was written in the 1970‘s. I’m just not that masochistic. My intention was: install the IDE, install the language plugins, test debugging for a Hello World app in each language. That was the intention. I’m that dumb.

My hopes were that I could use one IDE for all the languages I play or experiment with: Erlang, Clojure, Scala, Java, Python, PHP, Ruby and C#. Having used Intellij a couple of years back I thought I’d give that the first try. It was that or Eclipse and I remember Eclipse being a bit unwieldy when I last used it with Erlang.

Installing in Intellij/IDEA was dead easy. Plugins are found by clicking ‘Preferences -> Plugins’ from the main menu. There is an Erlang plugin so I installed that. An immediate irritation is that you have to set the SDK manually. I’m not sure how hard it is to search $PATH for the erlc but it didn’t find it so I had to do it manually. Annoying. When compiling my test code I kept getting errors. “SDK not found” which I eventually nailed with Google (what did people do before Google & Stackoverflow?), then I would be unable to set any breakpoints. Then I found I needed to start the Erlang Port Mapper Daemon (epmd) by running “erl -sname foo -s init stop -noshell” in a shell. Even with the mapper daemon running it still wouldn’t do anything.

Running Clojure was equally painful: I found the plugin & installed it. Did a 2 line test program. Compiled it OK. Then when I ran it got a dialog saying ‘Clojure classes’ not attached to project’. So I had to find those, manually, add them manually by digging around in the config dialogs. Then I was still unable to debug by stopping on a breakpoint as all the options were greyed out. I think there is a jvm remote debug option to add ‘somewhere’. After may hours I gave up and tried the same experiment on Netbeans. Success out of the box!! It Just Worked™. But Oh, on Netbeans Scala didn’t work due to some weird XML related ‘make transitive’ error. PHP didnt work either. There was no Erlang plugin. But hey, OK, I now have a viable Java IDE. Intellij also didn’t do PHP though it seems that the core developers Jetbrains have realised there is a money making option in forking a dedicated PHP only version rather than relying on open source plugins from volunteers. Great. Except that for playing about with personal projects I’m not much minded to cough up money that could better be spend in the pub. As it was, after moaning on Twitter, I acquired a working copy of PHPStorm which installed fine and debugs perfectly. But it’s not free and I’d have like it to use PHP with Intellij/IDEA.

None of my development languages worked in Eclipse. Either there was no plugin, it didn’t work or it may have worked but in the maze of options I lost the will to live. Actually, I think Clojure may work in Eclipse but I’m not that sure & I’m too traumatised from the last attempt to go back and try again.

I didn’t find any solution for Ruby with any of these IDEs.

Conclusion

These are tools that ought to be doing the heavy lifting for me, but aren’t. The configuration of the systems was a nightmare & instead of having One IDE To Rule Them All I’ve now got three. I’ve done enough one man projects to know how hard it is get code to the point where it ships without problems when users play with it; I’m not too inclined to slag off the developers who are doing this stuff for free (because tightwads like me won’t pay) but god it’s frustrating. I suddenly realise the virtues of a good code editor like Textmate and a bash shell.

Update: for Netbeans & Scala apparently one removes “-make:transitive” from an option in nbproject/build-impl.xml