publications

F. Lauer and G. Bloch Hybrid System Identification: Theory and Algorithms for Learning Switching Models Springer, 2019 [Table of contents][Front matter] [Errata] Introduction What are Hybrid Systems? What is System Identification? Applications Outline of the Book System Identification Input-Output Models State-Space Models Recursive Identification Nonlinear System Identification Model Selection and Assessment Notes Classification Discrimination Clustering Notes Hybrid System Identification Hybrid System Models Identification Problems Other Related Problems Notes Exact Methods for Hybrid System Identification Straightforward Solutions Hardness Results Polynomial-Time Algorithms for Fixed Dimensions Global Optimization with Branch-and-Bound The Need for Approximation Schemes / Heuristics Notes Estimation of Switched Linear Models Fixed Number of Modes Free Number of Modes Notes Estimation of Pieceiwise Affine Models From Switched Affine Models to PWA Models From Nonlinear Models to PWA Models From Local Models to PWA Models Notes Recursive and State-Space Identification of Hybrid Systems Input-Output Models State-Space Models Notes Nonlinear Hybrid System Identification Continuous Optimization Approach for Switched Nonlinear Models Sparse Optimization Approach for Switched Nonlinear Models Sum-of-Norms Approach for Pieceiwse Smooth Models Notes Outlook Basics of Probability Basics of Linear Algebra

2024

F. Lauer
Margin theory for the scenario-based approach to robust optimization in high dimension
IEEE Transactions on Automatic Control, to appear, 2024
[abstract] This paper deals with the scenario approach to robust optimization. This relies on a random sampling of the possibly infinite number of constraints induced by uncertainties in the parameters of an optimization problem. Solving the resulting random program yields a solution for which the quality is measured in terms of the probability of violating the constraints for a random value of the uncertainties, typically unseen before. Another central issue is the determination of the sample complexity, i.e., the number of random constraints (or scenarios) that one must consider in order to guarantee a certain level of reliability. In this paper, we introduce the notion of margin to improve upon standard results in this field. In particular, using tools from statistical learning theory, we show that the sample complexity of a class of random programs does not explicitly depend on the number of variables. In addition, within the considered class, that includes polynomial constraints among others, this result holds for both convex and nonconvex instances with the same level of guarantees. We also derive a posteriori bounds on the probability of violation and sketch a regularization approach that could be used to improve the reliability of computed solutions on the basis of these bounds.

2022

L. Massucci, F. Lauer, and M. Gilson
A statistical learning perspective on switched linear system identification
Automatica, 145:110532, 2022
[abstract] Hybrid systems form a particularly rich class of dynamical systems based on the combination of multiple continuous subsystems and a discrete mechanism deciding which one of these is active at a given time. Their identification from input-output data involves nontrivial issues that were partly solved over the last twenty years thanks to numerous approaches. However, despite this effort, estimating the number of modes (or subsystems) of hybrid systems remains a critical and open issue. This paper focuses on switched linear systems and proposes an analysis of their identification based on statistical learning theory. This leads to new theoretically sound bounds on the prediction error of switched models on the one hand, and a practical method for the estimation of the number of modes on the other hand. The latter is inspired by the structural risk minimization principle developed in statistical learning for model selection. The proposed analysis is conducted under various assumptions on the model class and regularization schemes for which new algorithms are presented. Numerical experiments are also provided to illustrate the accuracy of the proposed method.

2021

R. Rouhi, M. Clausel, J. Oster, and F. Lauer
An interpretable hand-crafted feature-based model for atrial fibrillation detection
Frontiers in Physiology, 12:581, 2021
[abstract] Atrial Fibrillation (AF) is the most common type of cardiac arrhythmia. Early diagnosis of AF helps to improve therapy and prognosis. Machine Learning (ML) has been successfully applied to improve the effectiveness of Computer-Aided Diagnosis (CADx) systems for AF detection. Presenting an explanation for the decision made by an ML model is considerable from the cardiologists' point of view, which decreases the complexity of the ML model and can provide tangible information in their diagnosis. In this paper, a range of explanation techniques is applied to hand-crafted features based ML models for heart rhythm classification. We validate the impact of the techniques by applying feature selection and classification to the 2017 CinC/PhysioNet challenge dataset. The results show the effectiveness and efficiency of SHapley Additive exPlanations (SHAP) technique along with Random Forest (RF) for the classification of the Electrocardiogram (ECG) signals for AF detection with a mean F-score of 0.746 compared to 0.706 for a technique based on the same features based on a cascaded SVM approach. The study also highlights how this interpretable hand-crafted feature-based model can provide cardiologists with a more compact set of features and tangible information in their diagnosis.

2020

F. Lauer
Error bounds for piecewise smooth and switching regression
IEEE Transactions on Neural Networks and Learning Systems, 31(4):1183-1195, 2020
[abstract] The paper deals with regression problems, in which the nonsmooth target is assumed to switch between different operating modes. Specifically, piecewise smooth (PWS) regression considers target functions switching deterministically via a partition of the input space, while switching regression considers arbitrary switching laws. The paper derives generalization error bounds in these two settings by following the approach based on Rademacher complexities. For PWS regression, our derivation involves a chaining argument and a decomposition of the covering numbers of PWS classes in terms of the ones of their component functions and the capacity of the classifier partitioning the input space. This yields error bounds with a radical dependency on the number of modes. For switching regression, the decomposition can be performed directly at the level of the Rademacher complexities, which yields bounds with a linear dependency on the number of modes. By using once more chaining and a decomposition at the level of covering numbers, we show how to recover a radical dependency. Examples of applications are given in particular for PWS and swichting regression with linear and kernel-based component functions.

T.S. Illés, S.M. Jespersen, P. Reynders, F. Lauer, J.C. Le Huec, J.F. Dubousset
Axial plane characteristics of thoracic scoliosis and their usefulness for determining the fusion levels and the correction technique
European Spine Journal, 29:2000-2009, 2020
[abstract] Purpose There is insufficient information regarding axial plane characteristics of scoliosis despite its 3D nature. The posterior–anterior vertebral vector (VV) has been proposed to characterize the axial plane appearances of the thoracic scoliosis. This study aimed to highlight the importance of knowledge of axial plane features when determining fusion levels and correction techniques of thoracic curves. Methods Altogether, 233 thoracic curves were analyzed using the VV after proving its usability instead of 3D angles to determine axial plane parameters such as apical vertebral (APV) axial rotations, APV lateral displacement, and intervertebral rotations (IVR). K-means clustering and regression analysis were used to identify axial plane curve patterns and determine the relationship between the coronal angles and axial plane characteristics, respectively. Results A close correlation was found between 3D angles and VV projected angles. Eight axial plane clusters were distinct, exhibiting different lateral APV displacement toward the interacetabular axis with relatively small axial rotations and a simultaneous decrease in sagittal curves. The regression analysis showed that the correlation of coronal curve magnitude was significantly stronger (r = 0.78) with APV lateral translation than with APV axial rotation (r = 0.65). Conclusion Based on these findings, the primary goal of scoliosis correction should focus on minimizing lateral translation rather than eliminating axial rotation. Knowing the IVR in the axial plane helps accurately determine the limits of the structural curves. VV-based axial views can facilitate the accurate determination of the end vertebrae and selection of the appropriate correction technique of the curve.

2019

K. Musayeva, F. Lauer, and Y. Guermeur
Rademacher complexity and generalization performance of margin multi-category classifiers
Neurocomputing, 342:6-15, 2019
[abstract] One of the main open problems in the theory of multi-category margin classification is the form of the optimal dependency of a guaranteed risk on the number C of categories, the sample size m and the margin parameter γ. In this paper, under minimal learnability assumptions, we derive a new risk bound for multi-category margin classifiers. We improve the dependency on C over the state of the art when the margin loss function considered satisfies the Lipschitz condition. We start with the basic supremum inequality that involves the Rademacher complexity as a capacity measure. This capacity measure is then linked to the metric entropy through the chaining method. In this context, our improvement is based on the introduction of a new combinatorial metric entropy bound.

2018

F. Lauer
On the exact minimization of saturated loss functions for robust regression and subspace estimation
Pattern Recognition Letters, 112:317-323, 2018
[abstract] This paper deals with robust regression and subspace estimation and more precisely with the problem of minimizing a saturated loss function. In particular, we focus on computational complexity issues and show that an exact algorithm with polynomial time-complexity with respect to the number of data can be devised for robust regression and subspace estimation. This result is obtained by adopting a classification point of view and relating the problems to the search for a linear model that can approximate the maximal number of points with a given error. Approximate variants of the algorithms based on ramdom sampling are also discussed and experiments show that it offers an accuracy gain over the traditional RANSAC for a similar algorithmic simplicity.

F. Lauer
Global optimization for low-dimensional switching linear regression and bounded-error estimation
Automatica, 89:73-82, 2018
[abstract] The paper provides global optimization algorithms for two particularly difficult nonconvex problems raised by hybrid system identification: switching linear regression and bounded-error estimation. While most works focus on local optimization heuristics without global optimality guarantees or with guarantees valid only under restrictive conditions, the proposed approach always yields a solution with a certificate of global optimality. This approach relies on a branch-and-bound strategy for which we devise lower bounds that can be efficiently computed. In order to obtain scalable algorithms with respect to the number of data, we directly optimize the model parameters in a continuous optimization setting without involving integer variables. Numerical experiments show that the proposed algorithms offer a higher accuracy than convex relaxations with a reasonable computational burden for hybrid system identification. In addition, we discuss how bounded-error estimation is related to robust estimation in the presence of outliers and exact recovery under sparse noise, for which we also obtain promising numerical results.

F. Lauer
MLweb: A toolkit for machine learning on the web
Neurocomputing, 282:74-77, 2018
[website] [code] [abstract] This paper describes MLweb, an open source software toolkit for machine learning on the web. The specificity of MLweb is that all computations are performed on the client side without the need to send data to a third-party server. MLweb includes three main components: a JavaScript API for scientific computing (LALOLib), an extension of this library with machine learning tools (ML.js) and an online development environment (LALOLab) with many examples.

T.S. Illés, M. Burkus, S. Somoskeôy, F. Lauer, F. Lavaste, and J.F. Dubousset.
Axial plane dissimilarities of two identical Lenke-type 6C scoliosis cases visualized and analyzed by vertebral vectors
European Spine Journal, 27(9):2120-2129, 2018
[abstract] Purpose. The global appearance of scoliosis in the horizontal plane is not really known. Therefore, the aims of this study were to analyze scoliosis in the horizontal plane using vertebral vectors in two patients classified with the same Lenke group, and to highlight the importance of the information obtained from these vertebral vector-based top-view images in clinical practice. Methods. Two identical cases of scoliosis were selected, based on preoperative full-body standing anteroposterior and lateral radiographs obtained by the EOS™ 2D/3D system. Three-dimensional (3D) surface reconstructions of the spinal curves were performed by using sterEOS™ 3D software before and after surgery. In both patients, we also determined the vertebral vectors and horizontal plane coordinates for analyzing the curves mathematically before and after surgery. Results. Despite the identical appearance of spinal curves in the frontal and sagittal planes, the horizontal views seemed to be significantly different. The vertebral vectors in the horizontal plane provided different types of parameters regarding scoliosis and the impact of surgical treatment: reducing lateral deviations, achieving harmony of the curves in the sagittal plane, and reducing rotations in the horizontal plane. Conclusions. Vertebral vectors allow the evolution of scoliosis curve projections in the horizontal plane before and after surgical treatment, along with representation of the entire spine. The top view in the horizontal plane is essential to completely evaluate the scoliosis curves, because, despite the similar representations in the frontal and sagittal planes, the occurrence of scoliosis in the horizontal plane can be completely different.

2017

T.S. Illés, M. Burkus, S. Somoskeôy, F. Lauer, F. Lavaste, and J.F. Dubousset.
The horizontal plane appearances of scoliosis: what information can be obtained from top-view images?
International Orthopaedics, 41(11):2303-2311, 2017
[abstract] Purpose. A posterior-anterior vertebral vector is proposed to facilitate visualization and understanding of scoliosis. The aim of this study was to highlight the interest of using vertebral vectors, especially in the horizontal plane, in clinical practice. Methods. We used an EOS two-/three-dimensional (2D/3D) system and its sterEOS 3D software for 3D reconstruction of 139 normal and 814 scoliotic spines—of which 95 cases were analyzed pre-operatively and post-operatively, as well. Vertebral vectors were generated for each case. Vertebral vectors have starting points in the middle of the interpedicular segment, while they are parallel to the upper plate, ending in the middle of the segment joining the anterior end plates points, thus defining the posterior-anterior axis of vertebrae. To illustrate what information could be obtained from vertebral vector-based top-view images, representative cases of a normal spine and a thoracic scoliosis are presented. Results. For a normal spine, vector projections in the transverse plane are aligned with the posterior-anterior anatomical axis. For a scoliotic spine, vector projections in the horizontal plane provide information on the lateral decompensation of the spine and the lateral displacement of vertebrae. In the horizontal plane view, vertebral rotation and projections of the sagittal curves can also be analyzed simultaneously. Conclusions. The use of posterior-anterior vertebral vector facilitates the understanding of the 3D nature of scoliosis. The approach used is simple. These results are sufficient for a first visual analysis furnishing significant clinical information in all three anatomical planes. This visualization represents a reasonable compromise between mathematical purity and practical use.

2016

F. Lauer
On the complexity of switching linear regression
Automatica, 74:80-83, 2016
[abstract] This technical note extends recent results on the computational complexity of globally minimizing the error of piecewise-affine models to the related problem of minimizing the error of switching linear regression models. In particular, we show that, on the one hand the problem is NP-hard, but on the other hand, it admits a polynomial-time algorithm with respect to the number of data points for any fixed data dimension and number of modes.

2015

F. Lauer
On the complexity of piecewise affine system identification
Automatica, 62:148-153, 2015
[abstract] The paper provides results regarding the computational complexity of hybrid system identification. More precisely, we focus on the estimation of piecewise affine (PWA) maps from input-output data and analyze the complexity of computing a global minimizer of the error. Previous work showed that a global solution could be obtained for continuous PWA maps with a worst-case complexity exponential in the number of data. In this paper, we show how global optimality can be reached for a slightly more general class of possibly discontinuous PWA maps with a complexity only polynomial in the number of data, however with an exponential complexity with respect to the data dimension. This result is obtained via an analysis of the intrinsic classification subproblem of associating the data points to the different modes. In addition, we prove that the problem is NP-hard, and thus that the exponential complexity in the dimension is a natural expectation for any exact algorithm.

F. Lauer and H. Ohlsson
Finding sparse solutions of polynomial systems of equations via group sparsity optimization
Journal of Global Optimization, 62(2):319-349, 2015
[code] [abstract] The paper deals with the problem of finding sparse solutions to systems of polynomial equations possibly perturbed by noise. In particular, we show how these solutions can be recovered from group-sparse solutions of a derived system of linear equations. Then, two approaches are considered to find these group-sparse solutions. The first one is based on a convex relaxation resulting in a second-order cone programming formulation which can benefit from efficient reweighting techniques for sparsity enhancement. For this approach, sufficient conditions for the exact recovery of the sparsest solution to the polynomial system are derived in the noiseless setting, while stable recovery results are obtained for the noisy case. Though lacking a similar analysis, the second approach provides a more computationally efficient algorithm based on a greedy strategy adding the groups one-by-one. With respect to previous work, the proposed methods recover the sparsest solution in a very short computing time while remaining at least as accurate in terms of the probability of success. This probability is empirically analyzed to emphasize the relationship between the ability of the methods to solve the polynomial system and the sparsity of the solution.

2014

V.L. Le, F. Lauer, and G. Bloch
Selective l₁ minimization for sparse recovery
IEEE Transactions on Automatic Control, 59(11):3008-3013, 2014
[abstract] Motivated by recent approaches to switched linear system identification based on sparse optimization, the paper deals with the recovery of sparse solutions of underdetermined systems of linear equations. More precisely, we focus on the associated convex relaxation where the l₁-norm of the vector of variables is minimized and propose a new iteratively reweighted scheme in order to improve the conditions under which this relaxation provides the sparsest solution. We prove the convergence of the new scheme and derive sufficient conditions for the convergence towards the sparsest solution. Experiments show that the new scheme significantly improves upon the previous approaches for compressive sensing. Then, these results are applied to switched system identification.

T. Pham Dinh, H. M. Le, H. A. Le Thi, and F. Lauer
A difference of convex functions algorithm for switched linear regression
IEEE Transactions on Automatic Control, 59(8):2277-2282, 2014
[abstract] This technical note deals with switched linear system identi- fication and more particularly aims at solving switched linear regression problems in a large-scale setting with both numerous data and many parameters to learn. We consider the recent minimum-of-error framework with a quadratic loss function, in which an objective function based on a sum of minimum errors with respect to multiple submodels is to be minimized. The technical note proposes a new approach to the optimization of this nonsmooth and nonconvex objective function, which relies on Difference of Convex (DC) functions programming. In particular, we formulate a proper DC decomposition of the objective function, which allows us to derive a computationally efficient DC algorithm. Numerical experiments show that the method can efficiently and accurately learn switching models in large dimensions and from many data points.

2013

F. Lauer,
Estimating the probability of success of a simple algorithm for switched linear regression
Nonlinear Analysis: Hybrid Systems, 8:31-47, 2013
[code] [online demo] [abstract] This paper deals with the switched linear regression problem inherent in hybrid system identification. In particular, we discuss k-LinReg, a straightforward and easy to implement algorithm in the spirit of k-means for the nonconvex optimization problem at the core of switched linear regression, and focus on the question of its accuracy on large data sets and its ability to reach global optimality. To this end, we emphasize the relationship between the sample size and the probability of obtaining a local minimum close to the global one with a random initialization. This is achieved through the estimation of a model of the behavior of this probability with respect to the problem dimensions. This model can then be used to tune the number of restarts required to obtain a global solution with high probability. Experiments show that the model can accurately predict the probability of success and that, despite its simplicity, the resulting algorithm can outperform more complicated approaches in both speed and accuracy.

2011

V.L. Le, G. Bloch, and F. Lauer
Reduced-size kernel models for nonlinear hybrid system identification
IEEE Transactions on Neural Networks, 22(12):2398-2405, 2011
[abstract] This brief paper focuses on the identification of nonlinear hybrid dynamical systems, i.e., systems switching between multiple nonlinear dynamical behaviors. Thus the aim is to learn an ensemble of submodels from a single set of input- output data in a regression setting with no prior knowledge on the grouping of the data points into similar behaviors. To be able to approximate arbitrary nonlinearities, kernel submodels are considered. However, in order to maintain efficiency when applying the method to large data sets, a preprocessing step is required in order to fix the submodel sizes and limit the number of optimization variables. This brief paper proposes four approaches, respectively inspired by the fixed-size least-squares support vector machines, the feature vector selection method, the kernel principal component regression and a modification of the latter, in order to deal with this issue and build sparse kernel submodels. These are compared in numerical experiments, which show that the proposed approach achieves the simultaneous classification of data points and approximation of the nonlinear behaviors in an efficient and accurate manner.

F. Lauer and Y. Guermeur,
MSVMpack: a Multi-Class Support Vector Machine package
Journal of Machine Learning Research, 12:2269-2272, 2011
[code] [abstract] This paper describes MSVMpack , an open source software package dedicated to our generic model of multi-class support vector machine. All four multi-class support vector machines (M-SVMs) proposed so far in the literature appear as instances of this model. MSVMpack provides for them the first unified implementation and offers a convenient basis to develop other instances. This is also the first parallel implementation for M-SVMs. The package consists in a set of command-line tools with a callable library. The documentation includes a tutorial, a user’s guide and a developer’s guide.

F. Lauer, G. Bloch, and R. Vidal,
A continuous optimization framework for hybrid system identification
Automatica 47(3):608-613, 2011
[code] [abstract] We propose a new framework for hybrid system identification, which relies on continuous optimization. This framework is based on the minimization of a cost function that can be chosen as either the minimum or the product of loss functions. The former is inspired by traditional estimation methods, while the latter is inspired by recent algebraic and support vector regression approaches to hybrid system identification. In both cases, the identification problem is recast as a continuous optimization program involving only the real parameters of the model as variables, thus avoiding the use of discrete optimization. This program can be solved efficiently by using standard optimization methods even for very large data sets. In addition, the proposed framework easily incorporates robustness to different kinds of outliers through the choice of the loss function.

2008

F. Lauer and G. Bloch,
Incorporating prior knowledge in Support Vector Regression
Machine Learning 70(1):89-118, 2008
[abstract] This paper explores the incorporation of prior knowledge in support vector re- gresion by the addition of constraints. Equality and inequality constraints are studied with the corresponding types of prior knowledge that can be considered for the method. These include particular points with known values, prior knowledge on any derivative of the function either provided by a prior model or available only at some specific points and bounds on the function or any derivative in a given domain. Moreover, a new method for the simultaneous approximation of multiple outputs linked by some prior knowledge is proposed. This method also allows consideration of different types of prior knowledge on single outputs while training on multiple outputs. Synthetic examples show that incorporating a wide variety of prior knowledge becomes easy, as it leads to linear programs, and helps to improve the approximation in difficult cases. The benefits of the method are finally shown on a real- life application, the estimation of in-cylinder residual gas fraction in spark ignition engines, which is representative of numerous situations met in engineering.

F. Lauer and G. Bloch,
Incorporating prior knowledge in Support Vector Machines for classification: a review
Neurocomputing 71(7-9):1578-1594, 2008
[abstract] For classification, support vector machines (SVMs) have recently been introduced and quickly became the state of the art. Now, the incorporation of prior knowledge into SVMs is the key element that allows to increase the performance in many applications. This paper gives a review of the current state of research regarding the incorporation of two general types of prior knowledge into SVMs for classification. The particular forms of prior knowledge considered here are presented in two main groups: class-invariance and knowledge on the data. The first one includes invariances to transformations, to permutations and in domains of input space, whereas the second one contains knowledge on unlabeled data, the imbalance of the training set or the quality of the data. The methods are then described and classified into the three categories that have been used in literature: sample methods based on the modification of the training data, kernel methods based on the modification of the kernel and optimization methods based on the modification of the problem formulation. A recent method, developed for support vector regression, considers prior knowledge on arbitrary regions of the input space. It is exposed here when applied to the classification case. A discussion is then conducted to regroup sample and optimization methods under a regularization framework.

G. Bloch, F. Lauer, G. Colin, and Y. Chamaillard,
Support Vector Regression from simulation data and few experimental samples
Information Sciences 178(20):3813-3827, 2008
[abstract] This paper considers nonlinear modeling based on a limited amount of experimental data and a simulator built from prior knowledge. The problem of how to best incorporate the data provided by the simulator, possibly biased, into the learning of the model is addressed. This problem, although particular, is very representative of numerous situations met in engine control, and more generally in engineering, where complex models, more or less accurate, exist and where the experimental data which can be used for calibration are difficult or expensive to obtain. The first proposed method constrains the function to fit to the values given by the simulator with a certain accuracy, allowing to take the bias of the sim- ulator into account. The second method constrains the derivatives of the model to fit to the derivatives of a prior model previously estimated on the simulation data. The combination of these two forms of prior knowledge is also possible and considered. These approaches are implemented in the linear programming support vector regression (LP-SVR) framework by the addition, to the optimization problem, of constraints, which are linear with respect to the parameters. Tests are then performed on an engine control application, namely, the estimation of the in-cylinder residual gas fraction in Spark Ignition (SI) engine with Variable Camshaft Timing (VCT). Promising results are obtained on this application. The experiments have also shown the importance of adding potential support vectors in the model when using Gaussian RBF kernels with very few training samples.

2007

F. Lauer, C. Y. Suen, and G. Bloch,
A trainable feature extractor for handwritten digit recognition
Pattern Recognition 40(6):1816-1824, 2007
[abstract] This article focuses on the problems of feature extraction and the recognition of handwritten digits. A trainable feature extractor based on the LeNet5 convolutional neural network architecture is introduced to solve the first problem in a black box scheme without prior knowledge on the data. The classification task is performed by support vector machines to enhance the generalization ability of LeNet5. In order to increase the recognition rate, new training samples are generated by affine transformations and elastic distortions. Experiments are performed on the well-known MNIST database to validate the method and the results show that the system can outperform both SVMs and LeNet5 while providing performances comparable to the best performance on this database. Moreover, an analysis of the errors is conducted to discuss possible means of enhancement and their limitations.

2006

F. Lauer and G. Bloch,
Ho-Kashyap classifier with early stopping for regularization
Pattern Recognition Letters 27(9):1037:1044, 2006
[abstract] This paper focuses on linear classification using a fast and simple algorithm known as the Ho–Kashyap learning rule (HK). In order to avoid overfitting and instead of adding a regularization parameter in the criterion, early stopping is introduced as a regularization method for HK learning, which becomes HKES (Ho–Kashyap with early stopping). Furthermore, an automatic procedure, based on the generalization error estimation, is proposed to tune the stopping time. The method is then tested and compared to others (including SVM and LSVM), that use either l₁ or l₂-norm of the errors, on well-known benchmarks. The results show the limits of early stopping for regularization with respect to the generalization error estimation and the drawbacks of low level hyperparameters such as a number of iterations.

F. Lauer and H. Ohlsson
Sparse phase retrieval via group-sparse optimization
arXiv preprint, arXiv:1402.5803, 2014
[abstract] This paper deals with sparse phase retrieval, i.e., the problem of estimating a vector from quadratic measurements under the assumption that few components are nonzero. In particular, we consider the problem of finding the sparsest vector consistent with the measurements and reformulate it as a group-sparse optimization problem with linear constraints. Then, we analyze the convex relaxation of the latter based on the minimization of a block l1-norm and show various exact recovery and stability results in the real and complex cases. Invariance to circular shifts and reflections are also discussed for real vectors measured via complex matrices.

Y. Guermeur and F. Lauer,
A generic approach to biological sequence segmentation problems, application to protein secondary structure prediction
In: Pattern Recognition in Computational Molecular Biology: Techniques and Approaches, M. Elloumi, C.S. Iliopoulos, J.T.L. Wang and A.Y. Zomaya (Eds.), Wiley, 2016

G. Bloch, F. Lauer and G. Colin,
On learning machines in engine control
In: Computational Intelligence in Automotive Applications, D. Prokhorov (Ed.), vol. 132 of Studies in Computational Intelligence, Springer, 2008

2021

L. Massucci, F. Lauer and M. Gilson
Regularized switched system identification: a Statistical learning perspective
Proc. of the 7th IFAC Conf. on Analysis and Design of Hybrid Systems (ADHS), 2021
[abstract] Switched system identification is a challenging problem, for which many methods were proposed over the last twenty years. Despite this effort, estimating the number of modes of switched systems from input–output data remains a nontrivial and critical issue for most of these methods. This paper discusses a recently proposed statistical learning approach to deal with this issue and proposes to go one step further by considering new results dedicated to regularized models. Optimization algorithms devised to tackle the estimation of such models from data are also proposed and illustrated in a few numerical experiments.

L. Massucci, F. Lauer and M. Gilson
How statistical learning can help to estimate the number of modes in switched system identification?
Proc. of the 19th IFAC Symposium on System Identification (SYSID), 2021
[abstract] This paper deals with hybrid dynamical system identification, and focuses more particularly on the estimation of the number of modes. An evaluation of a recent method based on model selection techniques from statistical learning is proposed, together with its comparison with more standard approaches based on algebraic arguments. Overall, three methods are benchmarked in various settings, including different noise conditions and data set sizes. The results provide insights into the respective advantages and weaknesses of the methods, thus yielding a set of guidelines on the choice of the most suitable method in a given situation for the practitioner.

2020

F. Lauer
Risk bounds for learning multiple components with permutation-invariant losses
AISTATS, 2020
[abstract] This paper proposes a simple approach to derive efficient error bounds for learning multiple components with sparsity-inducing regularization. Indeed, we show that for such regularization schemes, known decompositions of the Rademacher complexity over the components can be used in a more efficient manner to result in tighter bounds without too much effort. We give examples of application to switching regression and center-based clustering/vector quantization. Then, the complete workflow is illustrated on the problem of subspace clustering, for which decomposition results were not previously available. For all these problems, the proposed approach yields risk bounds with mild dependencies on the number of components and completely removes this dependency for nonconvex regularization schemes that could not be handled by previous methods. Erratum : the last formula of the proof of Theorem 3 in Appendix C should show norms of x_i to the power 4 instead of 2. As a result, the Frobenius norm of X appearing in Theorems 3-5 should be the norm of a vector with each component equal to the squared norm of an x_i.

L. Massucci, F. Lauer and M. Gilson
Structural risk minimization for switched system identification
Proc. of the 59th IEEE Conf. on Decision and Control (CDC), 2020
[abstract] This paper deals with the identification of hybrid dynamical systems that switch arbitrarily between modes. In particular, we focus on the critical issue of estimating the number of modes. A novel method inspired by model selection techniques in statistical learning is proposed. Specifically, the method implements the structural risk minimization principle, which relies on the minimization of an upper bound on the expected prediction error of the model. This so-called generalization error bound is first derived for static switched systems using Rademacher complexities. Then, it is extended to handle non independent observations from a single trajectory of a dynamical system. Finally, it is further tailored to the needs of model selection via a uniformization step. An illustrative example of the behavior of the method and its ability to recover the true number of modes is presented.

2018

K. Musayeva, F. Lauer and Y. Guermeur,
A sharper bound on the Rademacher complexity of margin multi-category classifiers
In Proc. of the 26th Eur. Symp. on Artificial Neural Networks (ESANN), pp. 503-508, 2018

F. Lauer,
Error bounds with almost radical dependence on the number of components for multi-category classification, vector quantization and switching regression
In French Conference on Machine learning (FCML/CAp), 2018

2017

K. Musayeva, F. Lauer and Y. Guermeur,
Metric entropy and Rademacher complexity of margin multi-category classifiers
In Proc. of the 26th Int. Conf. on Artificial Neural Networks (ICANN), 2017

2015

E. Didiot and F. Lauer,
Efficient optimization of multi-class support vector machines with MSVMpack
In Modelling, Computation and Optimization in Information Systems and Management Sciences, Proc. of MCO 2015, pp. 23-34, 2015

2014

F. Lauer and G. Bloch,
Piecewise smooth system identification in reproducing kernel Hilbert space
In Proc. of the 53rd IEEE Conf. on Decision and Control (CDC), Los Angeles, CA, USA, pp. 6498-6503, 2014

2013

V.L. Le, F. Lauer, L. Bako, and G. Bloch,
Learning nonlinear hybrid systems: from sparse optimization to support vector regression
In Proc. of the 16th ACM Int. Conf. on Hybrid Systems: Computation and Control (HSCC), Philadelphia, PA, USA, pp. 33-42, 2013

V.L. Le, F. Lauer, and G. Bloch,
Identification of linear hybrid systems: a geometric approach
In Proc. of the American Control Conference (ACC), Washington, DC, USA, pp. 830-835, 2013

L. Bako, V.L. Le, F. Lauer, and G. Bloch,
Identification of MIMO switched state-space models
In Proc. of the American Control Conference (ACC), Washington, DC, USA, pp. 71-76, 2013

2012

F. Thomarat, F. Lauer, and Y. Guermeur,
Cascading discriminant and generative models for protein secondary structure prediction
In Proc. of the 7th IAPR Int. Conf. on Pattern Recognition in Bioinformatics (PRIB), Tokyo, Japan, vol. 7632 of LNCS (LNBI), pp. 166-177, 2012

F. Lauer, V.L. Le, and G. Bloch,
Learning smooth models of nonsmooth functions via convex optimization
In Proc. of the 22nd IEEE Int. Workshop on Machine Learning for Signal Processing (MLSP), Santander, Spain, 2012

2010

F. Lauer, G. Bloch, and R. Vidal,
Nonlinear hybrid system identification with kernel models
In Proc. of the 49th IEEE Int. Conf. on Decision and Control (CDC), Atlanta, GA, USA, pp. 696-701, 2010

2009

F. Lauer and C. Schnörr,
Spectral clustering of linear subspaces for motion segmentation
In Proc. of the 12th IEEE Int. Conf. on Computer Vision (ICCV), Kyoto, Japan, pp. 678-685, 2009

F. Lauer, R. Vidal, and G. Bloch,
A product-of-errors framework for linear hybrid system identification
In Proc. of the 15th IFAC Symp. on System Identification (SYSID), Saint-Malo, France, pp. 563-568, 2009

2008

F. Lauer and G. Bloch,
Switched and piecewise nonlinear hybrid system identification
In Proc. of the 11th Int. Conf. on Hybrid Systems: Computation and Control (HSCC), St-Louis, MO, USA, vol. 4981 of LNCS, pp. 330-343, 2008

F. Lauer and G. Bloch,
A new hybrid system identification algorithm with automatic tuning
In Proc. of the 17th IFAC World Congress, Seoul, Korea, pp. 10207-10212, 2008

2007

G. Bloch, F. Lauer, G. Colin, and Y. Chamaillard,
Combining experimental data and physical simulation models in support vector learning
In Proc. of the 10th Int. Conf. on Eng. Appl. of Neural Networks (EANN), Thessaloniki, Greece, vol. 284 of CEUR-WS.org, pp. 284-295, 2007

2006

F. Lauer and G. Bloch,
Méthodes SVM pour l'identification
In Proc. of the Journées pour l'Identification et la Modélisation Expérimentale (JIME), Poitiers, France, 2006 (french)

2004

F. Lauer, M. Bentoumi, G. Bloch, G. Millérioux, and P. Aknin,
Ho-Kashyap with early stopping vs. soft-margin SVM for linear classifiers: an application
In Advances in Neural Networks - ISNN 2004, Dalian, China, vol. 3173 of LNCS, pp. 524-530, 2004

F. Lauer,
Optimization and statistical learning theory for piecewise smooth and switching regression
Habilitation thesis, Université de Lorraine, France, 2019

F. Lauer,
From support vector machines to hybrid system identification
PhD thesis, Université Henri Poincaré Nancy 1, France, 2008

F. Lauer,
Increasing the recognition rate of handwritten digit classifiers
Rapport de DEA, Université Henri Poincaré - Nancy 1, France, 2005, also published as a Technical Report, CENPARMI, Concordia University, Canada, 2005