Tag Archives: genetic programming

Linux beginnings & straight into Docker containers

Now my friend Ed (see link below, no sponsored links so far), has been encouraging me to get on with Linux for a while.

Link to Ed’s Yeti tool, a portable CNC router here: https://www.yetitool.com/products/product-details/smartbench

Last night, I was letting him know I was making progress… my wife, Ed and I all shared the easy joke that Linux looks like the Matrix. It’s also the future, and free.

19 10 01 BLOGGED Docker-Logo-White-RGB_Vertical-BG_0

Ed gave me a Raspberry Pi a while back. Not got to that recently.

What I have been doing, for the CondorGP project, is getting containers working, and immediately trying WineHQ, as I want to emulate Amibroker – classically and annoyingly, it’s a brilliant but Windows only program.

WineHQ highlights Amibroker as Silver on the current version: https://appdb.winehq.org/objectManager.php?sClass=application&iId=1084


So far so good. I can do all sorts of basics with Docker – using Docker Desktop (as I have a Windows machine and prefer not to boot back and forth the whole time).

I have WineHQ installed, and just trying to get Amibroker to actually run. Won’t be long I’m sure.

Next stop:

Kubernetes to run many containers, probably via Confluent, on AWS. That said, IBM conversations continue and Kubernetes would work there too. The major point of transferring operational ability to Linux and containerising it is to allow and IBM run too, if I decide we want to.




Seeking developers for crazy ambitious project

This set of questions was provided by KK on https://www.meetup.com/techstartup2019/ and is a useful set of initial requirements & information based on where this CondorGP project is.

Have a look through the rest of the blog, if you are interested. Links etc below.

Thanks for reading.

Cheers, Hugh19 10 01 BLOGGED Meetup KK Group


Brief description of your company / business / what you need the developer to build (and anything important that a developer should understand about the work):

See previous blog link: https://allthingsposco.com/2019/06/04/condor-gp-aims-1/


Technology stack requirements (if you need advice on this, or are flexible, just state “open” here):


We will need to be flexible throughout our journey, and only knowing one part of the tech sphere and not being able to move will be antithesis of our approach.

A key skill requirement is the ability to quickly pick up other new languages, effectively and quickly (due time, effort and time expected).

Some information here too, see previous link here: https://allthingsposco.com/2019/09/10/who-the-team-needs-not-default-male/


Time requirement (part-time, full-time, freelancer, permanent):

Part time freelancer ideally, later moving to be more permanent and full time as the project develops


Payment preference (cash, cash / equity, etc.):

Equity only initially, with strong potential for considerable cash later, once funded.

Good potential of additional work on other projects through my network too.


Preferred start date:

ASAP for the right candidate


Contact details (email, mobile etc.):

+44 (0) 7976 151 725



This blog: www.allthingsposco.com

Who the team needs: not Default Male

Not Default Male

I am – and I’ll say this, firmly and once – English, privileged, white, male. 38 years, from a decent family. That makes me Default Male. So in this current day and age, it is correct to compensate in the other direction(s). For more on Default Male – see Grayson’s book – the Descent of Man19 09 10 The Descent of Man - Grayson Perry - BLOGGED.

What the project and firm really needs is someone not at all like me.

And with ability to learn and be good at all sorts of stuff. Coding in general, multi-tasking, communication, being a decent human. The list is not really technical in focus, but technicals must be a strong part of it.

Super moral and demanding and ambitious

CondorGP’s aim of running ecosystems of trading-strategy-survival-machines may seem purely old-school, go make money fast and don’t care in it’s attitude. The moral angle is strong when we are clear about the long term, the why we do stuff.

Being able to start and run an effective finance AI firm is one thing. Knowing how to behave if it does work is quite another. And if Posco Consulting and CondorGP do really well, where will we take it.

To a place where we contribute heavily to changing the game, and contributing.

TBC on how, though as the founder, I’m full of ideas.


CondorGP works on / with the following:

  1. Java (for ECJ, the Evolutionary Computation in Java package)
  2. Windows and Linux (for day to day, and containerisation)
  3. Cloud (AWS, IBM)
  4. Eclipse, Git, Maven, Bitbucket
  5. Docker, Kubernetes
  6. Trello, Slack
  7. Amibroker, Interactive Brokers

Prediction is chancy: Selfish Gene 1

Page 55, Selfish Gene new edition.

“Prediction in a complex world is a chancy business. Every decision that a survival machine takes is a gamble, and it is the business of genes to program brains in advance so that on average they take decisions that pay off. The currency used in the casino of evolution is survival, strictly gene survival, but for many purposes individual survival is a reasonable approximation… ”

Continue reading Prediction is chancy: Selfish Gene 1

Bit zero on Genetic Programming

When I mention genetic programming the faces of those I talk to often go a bit funny, scrunched up. They don’t get it or can’t easily understand. Fair enough. Let me try and put it simply.

What does genetic programming achieve?

Genetic programming is known for producing unusual and successful outcomes. These are often computer programs, but can be physical objects too. The odd looking NASA antennae shown here was the first artificially evolved object in space, and achieved good transmission rates for low power consumption at odd angles as the satellite flew around Earth studying the magnetosphere in the mid 2000’s. 19 06 10 NASA GP Evolved Antennae BLOGGED.png

Obviously, it depends what the outcome is that the evolution is aimed at. This video from Karl Sims is from 1994 and shows simulated virtual creatures evolved to achieve a range of outcomes. This 2014 video from Cornell University shows ‘electrophysical robots’.

Genetic programming can be used to achieve all sorts of outcomes, provided that the fitness function can be programmed.

Where did genetic programming come from?

Evolutionary approaches were started in the 1960’s in Berlin and then in the 1990’s John Koza popularised the genetic programming technique at MIT. He published big fat heavy books on the topic, including Genetic Programming II, which I have on my bookshelf, and Genetic Programming Theory and Practice II which I also have hardcopy.

Genetic programming is: “Using code to evolve code to solve a problem”

This is the way I often put it. Genetic programming uses Darwinian evolutionary approaches coded into the computer, with a clear outcome specified. The outcome defines what the evolved code you get does.

To achieve genetic programming you need:

  1. A definition of what a good outcome is (the fitness function)
  2. DNA blocks (the small bits of code you will put into the evolutionary processes)
  3. A way of evolving the individuals (the code that manages the evolution process)

Throw them at the wall, and see what sticks

The fitness function is crucial. It defines how you will select from the evolved individuals. As each individual is evolved, they are ‘thrown at the wall’, or tested using the fitness function.
Those individuals that get ‘good’ scores are kept, and breeding happens more with higher them.


The process goes on, generation after generation, following that approach. And you get graphs like this one.Blogged IMG_2017_08_05 Website from CROC_Fig_015 2D Fitness Graph

Automatic Programming

Back in the 1990’s, a term often used was ‘Automatic Programming’, i.e. instead of developers working to code a solution to the problem, this technique outputs the program without a person directly involved. Of course a human must be involved to define and set up the problem.

The subtitle for one of those books above is “Automatic Discovery of Reusable Programs”, and this is what researchers really wanted to achieve. Put another way:

“Genetic programming addresses this challenge by providing a method for automatically creating a working computer program from a high-level problem statement of the problem”

For some more detail, see below:

Continue reading Bit zero on Genetic Programming

Condor GP technology

There are lots of reasons to for me to write about the technology we will use for CondorGP. So here goes, first pass. Bear in mind that this is as I see it now, and it’ll likely change.

Genetic Programming is the fundamental, underpinning technology application. I’ll write seperately about Genetic Programming in another post. Here I want to describe the technology stack the project is currently using.

Refresh on what CondorGP is

First, let’s start with a little of what the CondorGP project is. You can also, see the CondorGP aims, to refresh. They are high level, but give a picture.  CondorGP looks to use Genetic Programming to build an automated trading system.

What technology CondorGP uses

Here I will attempt to write simply and clearly, so that each technology is identified with a clear utility, to explain why each technology is used. In brackets is the current status of usage, today, e.g. (in use).

I’ll work from output down, from trading on the financial markets ‘downwards’.

  1. Interactive Brokers – a trading platform that will provide paper and real trading on financial markets (planned)
  2. Amibroker – a trading analysis tool, that provides data and allows testing of potential trading systems we have evolved (in use) NB. Headline image for this post is an Amibroker screenshot from amibroker.com, and not the way CondorGP will use Amibroker…
  3. Amazon Web Services or AWS – a massive cloud services platform, that allows processing of our genetic programming evolution and everything else (in use, nascent)
  4. Bit Bucket – Atlassian’s git code management platform, that provides pipelines for code to be tested, deployed and uplinked to AWS (in use)
  5. Git – distributed version control for software, allows sharing and co-working by teams on software projects (in use)
  6. Egit for the free Eclipse IDE (both in use)
  7. Jbehave – software project that enables BDD or Behaviour Driven Development to allow easy and accessible tests to be written and run (in use)
  8. Junit – to run the tests (in use, currently Junit 4)
  9. Maven – Apache’s method for control software project libraries, called dependencies (in use, via Eclipse’s Maven plugin)
  10. ECJ – the Evolutionary Computation in Java package, from George Mason University, that provides a powerful and wide ranging list of evolutionary capabilites (in use, version 22 from the maven repository)
  11. Jacob – the Java com bridge, currently limits the project to Jave version 8, see below. This enables Java to run Amibroker in Windows (in use)
  12. AWS toolkit for Eclipse – AWS integration for Eclipse, to help run AWS from the Java IDE (in use)
  13. Spring – the software development framework from Pivotal, to help with dependency injection and other coding paradigms (in use, 2.0.1.RELEASE)
  14. Java – the object-oriented software language that ECJ is written in, that we use to run ECJ, and write our tests, run AWS etc (in use, currently JDK 8u211 and JRE 8)

Condor GP aims (1)

This will be the first of a few posts about the CondorGP aims, as I hone my thinking, and how to express it.

Written once and with one edit:

  1. Demonstrate the power of genetic programming to deal with complex, non-linear data heavy problems, by trading on the financial markets
  2. Provide a positive income from this trading
  3. Provide this income in a stable way that can be perpetuated
  4. Achieve stable ongoing income by a design that minimises human input, hence minimising the subjectivity inherent in human input
  5. Provide this income for participants in our business model (e.g. investors, traders, licensees of our code)
  6. Provide this income for other parties, including myself and my family, and a broader socially focused group – in order to provide benefit to society
  7. Achieve these aims in a way that can be replicated so that we can multiply the process and achieve similar outcomes many times

I’ll follow up on the business model, of course, as we define it.

I’ll also follow up on the socially focused aims of CondorGP.

Condor GP: some of the inspiration

The CondorGP project came together over many years. Since 2005 in fact.

There are many pieces of the puzzle and my bet is that no-one else has cobbled together the bits I have, in the way that I have.

I should note there are some inspirational bits that I’m not going to talk about much (at all) as they are key to the design.

Self sustaining something

At one point at University, Alex, my coursemate and housemate and I were talking stuff. About how wouldn’t it be better if you could work on something, a business or an idea, and then when it was up and running then it would look after itself.

Seems minor, and obvious, but has influenced the thinking.

Genetic progammming

I came across Genetic Programming in an interesting textbook I bought at Brunel University, where Alex and I did an AI module as part of our engineering masters. This book was: Artificial Intelligence – A guide to intelligent systems, by Michael Negnevitsky

Alex and I worked together on an AI project, using ANNs, Artificial Neural Networks, like Deepmind used to create AlphaGo. AlphaGo came later, it must be said…

But flicking through again after University in 2005, I thought the sections on Genetic Programming were more interesting. I’ve still got the same copy of Negnevitsky’s book, so I type out directly:

“Genetic programming offers a solution to the main challenge of computer science – making computers solve problems without being explicitly programmed.”

This was pretty mind blowing and intriguing to a geeky engineering graduate. Further:

“Genetic programming represents an application of genetic model of learning to programming. Its goal is to evolve not a coded representation of some problem, but rather a computer code that solves the problem. That is, genetic programming generates computer programs as the solution.” 

So the first steps of inspiration where that the power of genetic programming was enormous.

Key point 1 being that Darwin had hit upon the idea that evolution as a mechanism that had allowed or encouraged, or maybe actually been the basis for all the complexity and wonder of our planet. Key point 2 being that something of that power could be made available to us as part of computer work to solve complex problems. I quote again:

“evolutionary strategies can solve a wide range of problems. They provide robust and reliable solutions for highly complex, non-linear search and optimisation problems that previously could not be solved at all (Holland, 1995, Schwefel, 1995).”

So my brain immediately went to solving the obvious problem to a new graduate, how to make money. And directly to trading on the stock market.

I’ll look to write about genetic programming itself, in future posts, to share a little of how this works.

Condor GP: my AI fintech project

See the basic explanation on poscoconsulting.com

This is to start off the blog about CondorGP.

CondorGP is an ambitious project I’ve had swirling slowly but surely around in my head since about 2005, gathering various useful inputs along the way.

In 2015, I started to take it more seriously. The first note I have of my long kept spreadsheet that is my notepad for everything (currently 16,900 lines long!) indicates that I started in on the 2nd of April 2015. 19 06 03 Blogged CROC_Fig_027b

Firstly, I did as simple as an idea as I could, called CROC, using Matlab to simulate the CROC situation and to provide a genetic programming package.

The image here shows one of the outputs from the Matlab CROC simulation. These graphs made me realise how much I enjoyed 3D charts, and trying to work out what they meant.

In my next post, I’ll write a bit about where the inspiration for this project came from.