:mod:`fitting` --- Model Fitting ******************************** .. module:: ailib.fitting .. contents :: Introduction ============ The term model is used in a very general manner. A model usually is a mathematical (often a statistical) tool that can be adjusted to some given data points. Once set up (trained), it can be evaluated at arbitrary input values. There are roughly two types of models: a) Classification b) Regression In classification problems an input point has to be assigned to a cluster. If the clusters are labeled, the collection of all clusters defines a codebook, so that the problem can also be viewed as finding the optimal code for each input value. Loosely speaking regression means curve fitting, i.e. adjust parameters of a mathematical function such that it optimally represents the training data. Of course, it has to be defined what optimal means. There is no general answer to this question, as it depends on the fitting target (the model). Often, the residual sum of squares is used, i.e. the sum of squared deviation between measurement and prediction from the model. To be clear, the model only specifies how training data is explained. For learning problems, the model is incomplete (some information is missing, e.g. parameters) and there's need for a learning algorithm that determines this information. The model learning algorithms strongly depend on the input their given. Basically, there are two situations: a) Supervised learning b) Unsupervised learning For supervised learning, the input values are measured together with the target output values. Both are presented to the fitting algorithm, so that it can train the model to fit the targets optimally. However, it's not always the case that targets are available. If there are only input but no output values, it's an unsupervised learning problem. This implies that it's not possible to use an error measurement for training or evaluation. So unsupervised learning algorithms try to find some (hidden) structure in the input data. A typical example is finding the location where the density of input points is maximal. Data model ========== There are three kinds of information: 1. Features 2. Labels 3. Weights A feature is the same as one input value. It may be a single value, however the inputs are often multidimensional, so in general the features are vectors (lists). The labels are the target values to the respective input. In most cases, the labels are unidimensional. However, the concrete data format (of input and output values) strongly depends on the model and thus is specified by the learning algorithm. Some algorithms also allow weighted samples. The weights are usually real numbered values, passed as a list to the learning algorithm. Usually, the training data is passed to the :func:`fit ` method seperately through the respective parameters. For unsupervised learning, the *labels* argument will be ignored. Internally, there are several ways to represent training data. For details, see :ref:`core-conversion`. In this library, the model and learning algorithm are combined in a single class. The class :class:`Model` defines the interface for all learning algorithms (supervised or unsupervised). .. autoclass:: Model :members: :show-inheritance: Many learning algorithms use several models to make the prediction more robust. For this, a very general interface is defined. The class :class:`Committee` allows to collect models and it provides different mixins for evaluation and error measurement. .. autoclass:: Committee :members: :show-inheritance: Learning algorithms =================== **Supervised learning** .. toctree:: :maxdepth: 1 linear nlsq kNN bagging boosting tree **Unsupervised learning** .. toctree:: :maxdepth: 1 clustering