Learning under Concept Drift

Splash image for post

It’s very rare for both the underlying generative processes that produce the raw data, and the systems we use to measure, transform, and store that raw data, to be static and unchanging. More commonly, they evolve: distributions shift, relationships and constraints between the different dimensions of the data drift and break, data stops being available, and assumptions about the semantics of certain attributes cease to be valid. Multiple of those shifts can happen on different timescales, and sometimes in abrupt ways.

This can have a profound effect on machine learning models. An underlying assumption of machine learning models is that the state of the world observed at training time is representative of the environment and time at which the model is deployed. When that assumption is invalid, in the best case scenario, predictive performance might be degraded; a monitoring component might be able to detect this regression, attribute it to a specific root cause, and trigger a retrain using a different dataset. In the worst case, it will continue to silently, happily serve non-sensical results.

Continue reading

Rarity of Jupiter-like planets means planetary systems exactly like ours may be scarce

[This short article I wrote has been published on The Conversation UK.]

Is our little corner of the galaxy a special place? As of this date, we’ve discovered more than 1,500 exoplanets: planets orbiting stars other than our sun. Thousands more will be added to the list in the coming years as we confirm planetary candidates by alternative, independent methods.

In the hunt for other planets, we’re especially interested in those that might potentially host life. So we focus our modern exoplanet surveys on planets that might be similar to Earth: low-mass, rocky and with just the right temperature to allow for liquid water. But what about the other planets in the solar system? The Copernican principle – the idea that the Earth and the solar system are not unique or special in the universe – suggests the architecture of our planetary system should be common. But it doesn’t seem to be.

Continue reading

AstroTRENDS: Weasel words

Credit: Cliff I added a bunch of new keywords to AstroTRENDS, mostly suggested by friends and people in the community who had read my Facebook post.

A thought I had yesterday is the following: has the astronomical literature become more speculative, and perhaps less committed to audacious claims, in recent times? It is difficult to test this hypothesis by merely querying ADS for abstract keywords. It would certainly be better served by a natural-language processing analysis of the full text, although this is just my uninformed speculation.

Continue reading

The Automated Planet Finder, Systemic and Super Planet Crash

[This short article I wrote has been published on The Conversation UK.]

The following is a short article about the Automated Planet Finder, Systemic and Super Planet Crash. We recently announced the first batch of exoplanets that were discovered in the first few months of science operation of APF. The first two systems (HD141399 and Gliese 687) have been submitted and will be available on astro-ph shortly.

Continue reading