Category Archives: Posco GP

Linux beginnings & straight into Docker containers

Now my friend Ed (see link below, no sponsored links so far), has been encouraging me to get on with Linux for a while.

Link to Ed’s Yeti tool, a portable CNC router here: https://www.yetitool.com/products/product-details/smartbench

Last night, I was letting him know I was making progress… my wife, Ed and I all shared the easy joke that Linux looks like the Matrix. It’s also the future, and free.

19 10 01 BLOGGED Docker-Logo-White-RGB_Vertical-BG_0

Ed gave me a Raspberry Pi a while back. Not got to that recently.

What I have been doing, for the CondorGP project, is getting containers working, and immediately trying WineHQ, as I want to emulate Amibroker – classically and annoyingly, it’s a brilliant but Windows only program.

WineHQ highlights Amibroker as Silver on the current version: https://appdb.winehq.org/objectManager.php?sClass=application&iId=1084

Progress:

So far so good. I can do all sorts of basics with Docker – using Docker Desktop (as I have a Windows machine and prefer not to boot back and forth the whole time).

I have WineHQ installed, and just trying to get Amibroker to actually run. Won’t be long I’m sure.

Next stop:

Kubernetes to run many containers, probably via Confluent, on AWS. That said, IBM conversations continue and Kubernetes would work there too. The major point of transferring operational ability to Linux and containerising it is to allow and IBM run too, if I decide we want to.

 

 

 

Seeking developers for crazy ambitious project

This set of questions was provided by KK on https://www.meetup.com/techstartup2019/ and is a useful set of initial requirements & information based on where this CondorGP project is.

Have a look through the rest of the blog, if you are interested. Links etc below.

Thanks for reading.

Cheers, Hugh19 10 01 BLOGGED Meetup KK Group

 

Brief description of your company / business / what you need the developer to build (and anything important that a developer should understand about the work):

See previous blog link: https://allthingsposco.com/2019/06/04/condor-gp-aims-1/

 

Technology stack requirements (if you need advice on this, or are flexible, just state “open” here):

Open.

We will need to be flexible throughout our journey, and only knowing one part of the tech sphere and not being able to move will be antithesis of our approach.

A key skill requirement is the ability to quickly pick up other new languages, effectively and quickly (due time, effort and time expected).

Some information here too, see previous link here: https://allthingsposco.com/2019/09/10/who-the-team-needs-not-default-male/

 

Time requirement (part-time, full-time, freelancer, permanent):

Part time freelancer ideally, later moving to be more permanent and full time as the project develops

 

Payment preference (cash, cash / equity, etc.):

Equity only initially, with strong potential for considerable cash later, once funded.

Good potential of additional work on other projects through my network too.

 

Preferred start date:

ASAP for the right candidate

 

Contact details (email, mobile etc.):

+44 (0) 7976 151 725

hugh.harford@poscoconsulting.com

www.poscoconsulting.com

This blog: www.allthingsposco.com

Help! Help! I am literally overflowing with ideas

This is a quick one, but doubtless I’ll come back to it.

I need help, not ‘save my life’ type, or anything like that. I need help of the ‘how to make real any of these ideas’ type help.

I need help to realise all these endless awesome ideas (which I’m firmly not going to write up, yet).

I need a unit, a squad, a loose form team.

We need to be only cohesive enough to agree to do something and then raggedy and determinedly do it. What ever, in that moment it might be.

I suggest the book pictured. A fine thing, worth re-reading many times.

Try reading this, and then try it’s suggestions and try them again. That is, if you want to.

Who the team needs: not Default Male

Not Default Male

I am – and I’ll say this, firmly and once – English, privileged, white, male. 38 years, from a decent family. That makes me Default Male. So in this current day and age, it is correct to compensate in the other direction(s). For more on Default Male – see Grayson’s book – the Descent of Man19 09 10 The Descent of Man - Grayson Perry - BLOGGED.

What the project and firm really needs is someone not at all like me.

And with ability to learn and be good at all sorts of stuff. Coding in general, multi-tasking, communication, being a decent human. The list is not really technical in focus, but technicals must be a strong part of it.

Super moral and demanding and ambitious

CondorGP’s aim of running ecosystems of trading-strategy-survival-machines may seem purely old-school, go make money fast and don’t care in it’s attitude. The moral angle is strong when we are clear about the long term, the why we do stuff.

Being able to start and run an effective finance AI firm is one thing. Knowing how to behave if it does work is quite another. And if Posco Consulting and CondorGP do really well, where will we take it.

To a place where we contribute heavily to changing the game, and contributing.

TBC on how, though as the founder, I’m full of ideas.

Technically

CondorGP works on / with the following:

  1. Java (for ECJ, the Evolutionary Computation in Java package)
  2. Windows and Linux (for day to day, and containerisation)
  3. Cloud (AWS, IBM)
  4. Eclipse, Git, Maven, Bitbucket
  5. Docker, Kubernetes
  6. Trello, Slack
  7. Amibroker, Interactive Brokers

Prediction is chancy: Selfish Gene 1

Page 55, Selfish Gene new edition.

“Prediction in a complex world is a chancy business. Every decision that a survival machine takes is a gamble, and it is the business of genes to program brains in advance so that on average they take decisions that pay off. The currency used in the casino of evolution is survival, strictly gene survival, but for many purposes individual survival is a reasonable approximation… ”


Continue reading Prediction is chancy: Selfish Gene 1

Bit zero on Genetic Programming

When I mention genetic programming the faces of those I talk to often go a bit funny, scrunched up. They don’t get it or can’t easily understand. Fair enough. Let me try and put it simply.

What does genetic programming achieve?

Genetic programming is known for producing unusual and successful outcomes. These are often computer programs, but can be physical objects too. The odd looking NASA antennae shown here was the first artificially evolved object in space, and achieved good transmission rates for low power consumption at odd angles as the satellite flew around Earth studying the magnetosphere in the mid 2000’s. 19 06 10 NASA GP Evolved Antennae BLOGGED.png

Obviously, it depends what the outcome is that the evolution is aimed at. This video from Karl Sims is from 1994 and shows simulated virtual creatures evolved to achieve a range of outcomes. This 2014 video from Cornell University shows ‘electrophysical robots’.

Genetic programming can be used to achieve all sorts of outcomes, provided that the fitness function can be programmed.

Where did genetic programming come from?

Evolutionary approaches were started in the 1960’s in Berlin and then in the 1990’s John Koza popularised the genetic programming technique at MIT. He published big fat heavy books on the topic, including Genetic Programming II, which I have on my bookshelf, and Genetic Programming Theory and Practice II which I also have hardcopy.

Genetic programming is: “Using code to evolve code to solve a problem”

This is the way I often put it. Genetic programming uses Darwinian evolutionary approaches coded into the computer, with a clear outcome specified. The outcome defines what the evolved code you get does.

To achieve genetic programming you need:

  1. A definition of what a good outcome is (the fitness function)
  2. DNA blocks (the small bits of code you will put into the evolutionary processes)
  3. A way of evolving the individuals (the code that manages the evolution process)

Throw them at the wall, and see what sticks

The fitness function is crucial. It defines how you will select from the evolved individuals. As each individual is evolved, they are ‘thrown at the wall’, or tested using the fitness function.
Those individuals that get ‘good’ scores are kept, and breeding happens more with higher them.

 

The process goes on, generation after generation, following that approach. And you get graphs like this one.Blogged IMG_2017_08_05 Website from CROC_Fig_015 2D Fitness Graph

Automatic Programming

Back in the 1990’s, a term often used was ‘Automatic Programming’, i.e. instead of developers working to code a solution to the problem, this technique outputs the program without a person directly involved. Of course a human must be involved to define and set up the problem.

The subtitle for one of those books above is “Automatic Discovery of Reusable Programs”, and this is what researchers really wanted to achieve. Put another way:

“Genetic programming addresses this challenge by providing a method for automatically creating a working computer program from a high-level problem statement of the problem”

For some more detail, see below:

Continue reading Bit zero on Genetic Programming

Condor GP technology

There are lots of reasons to for me to write about the technology we will use for CondorGP. So here goes, first pass. Bear in mind that this is as I see it now, and it’ll likely change.

Genetic Programming is the fundamental, underpinning technology application. I’ll write seperately about Genetic Programming in another post. Here I want to describe the technology stack the project is currently using.

Refresh on what CondorGP is

First, let’s start with a little of what the CondorGP project is. You can also, see the CondorGP aims, to refresh. They are high level, but give a picture.  CondorGP looks to use Genetic Programming to build an automated trading system.

What technology CondorGP uses

Here I will attempt to write simply and clearly, so that each technology is identified with a clear utility, to explain why each technology is used. In brackets is the current status of usage, today, e.g. (in use).

I’ll work from output down, from trading on the financial markets ‘downwards’.

  1. Interactive Brokers – a trading platform that will provide paper and real trading on financial markets (planned)
  2. Amibroker – a trading analysis tool, that provides data and allows testing of potential trading systems we have evolved (in use) NB. Headline image for this post is an Amibroker screenshot from amibroker.com, and not the way CondorGP will use Amibroker…
  3. Amazon Web Services or AWS – a massive cloud services platform, that allows processing of our genetic programming evolution and everything else (in use, nascent)
  4. Bit Bucket – Atlassian’s git code management platform, that provides pipelines for code to be tested, deployed and uplinked to AWS (in use)
  5. Git – distributed version control for software, allows sharing and co-working by teams on software projects (in use)
  6. Egit for the free Eclipse IDE (both in use)
  7. Jbehave – software project that enables BDD or Behaviour Driven Development to allow easy and accessible tests to be written and run (in use)
  8. Junit – to run the tests (in use, currently Junit 4)
  9. Maven – Apache’s method for control software project libraries, called dependencies (in use, via Eclipse’s Maven plugin)
  10. ECJ – the Evolutionary Computation in Java package, from George Mason University, that provides a powerful and wide ranging list of evolutionary capabilites (in use, version 22 from the maven repository)
  11. Jacob – the Java com bridge, currently limits the project to Jave version 8, see below. This enables Java to run Amibroker in Windows (in use)
  12. AWS toolkit for Eclipse – AWS integration for Eclipse, to help run AWS from the Java IDE (in use)
  13. Spring – the software development framework from Pivotal, to help with dependency injection and other coding paradigms (in use, 2.0.1.RELEASE)
  14. Java – the object-oriented software language that ECJ is written in, that we use to run ECJ, and write our tests, run AWS etc (in use, currently JDK 8u211 and JRE 8)