[bitL]

Google, Facebook & MS already have even automated research, i.e. automated selection of a loss function, network architecture, individualized network topology etc. Amazon is not there yet. The rest of industry is still in "stone age", just "considering" using something like AutoML for basic hyperparameter tuning.

[bitforger]

If you automate it, is it still research? Research implies some sort of hypothesis testing, yes?

I suppose OP means there will be two groups: people who use AutoML and people who try to make AutoML better.

[p1esk]

There should be at least 3 groups, because making AutoML better != making ML better.

[human_scientist]

Why? The concept of AutoML does include the design of novel algorithms.

[p1esk]

What do you mean? I thought AutoML was a tool to do neural architecture search, and hyperparameter tuning.

[human_scientist]

The field of automatic machine learning (abbreviated as AutoML) concerns all endeavours to automate the process of machine learning. To provide a sense of what could constitute AutoML, let me post a list from the "Call for Papers" of the International Workshop on Automatic Machine Learning (ICML 2018) [1]:

    * Model selection, hyper-parameter optimization, and model search
    * Neural architecture search
    * Meta learning and transfer learning
    * Automatic feature extraction / construction
    * Demonstrations (demos) of working AutoML systems
    * Automatic generation of workflows / workflow reuse
    * Automatic problem "ingestion" (from raw data and miscellaneous formats)
    * Automatic feature transformation to match algorithm requirements
    * Automatic detection and handling of skewed data and/or missing values
    * Automatic acquisition of new data (active learning, experimental design)
    * Automatic report writing (providing insight on automatic data analysis)
    * Automatic selection of evaluation metrics / validation procedures
    * Automatic selection of algorithms under time/space/power constraints
    * Automatic prediction post-processing and calibration
    * Automatic leakage detection
    * Automatic inference and differentiation
    * User interfaces and human-in-the-loop approaches for AutoML

[1] https://sites.google.com/site/automl2018icml/

[human_scientist]

> I don't see "Automatic design of novel algorithms" in this list. Can AutoML produce something as novel as a GAN, CapsNet, WaveNet, Transformer, Neural ODE, etc? Is that even considered to be one of its goals. In my opinion, there's a clear separation between a group of people trying to improve AutoML so that it's more useful in doing all those tasks on the list, and a group of people trying to invent next gen ML algorithms or DL architectures.

I agree with you from the view of the current state of the art methods and the current state of the AutoML / fundamental ML research communities. Current methods are very limited, but I can not think of a reason why a sufficiently general searchspace of architectures/pipelines could not produce something like a GAN or a WaveNet.

I do not think that designing algorithms as novel as the ones you listed is currently a goal of AutoML, as that is not something we have an attack for. However, I do think that with increasing capabilities, the field of AutoML will seek to automate every step of the machine learning pipeline - including the design of algorithms. E.g., once/if there are attacks to apply NAS for yielding truly novel architectures, I think NAS researchers will be happy to do just that -- wouldn't you call that AutoML then?

[p1esk]

I don't see "Automatic design of novel algorithms" in this list.

Can AutoML produce something as novel as a GAN, CapsNet, WaveNet, Transformer, Neural ODE, etc? Is that even considered to be one of its goals?

In my opinion, there's a clear separation between a group of people trying to improve AutoML so that it's more useful in doing all those tasks on the list, and a group of people trying to invent next gen ML algorithms or DL architectures.