WiDS Chemnitz 2022

Marika Kaden

(Hochschule Mittweida)

Nicole Mücke

(TU Braunschweig)

Anke Stoll

(FDTech Chemnitz)

Leena Chennuru Vankadara

(Universität Tübingen)

Franziska Diez

(Fraunhofer ITWM Kaiserslautern)

Text classification is one of the typical tasks in natural language processing. In Geman public administration this task arises from the fact that every municipality is obliged to entertain special messaging infrastructure, a so-called "besonderes elektronisches Behördenpostfach" (special electronic institutional post box). The received emails need to be forwarded to one of for up to 45 different offices. With the number of received messages expected to grow substantially in the near future, finding ways to forward at least a large share of them automatically is crucial. Furthermore, a high accuracy of the algorithm is important due to possible sensitive data. Because incoming messages will often lack structure, the classification algorithm needs to be based on content. Moreover, it must be able to deal with starkly different document lengths. One way to solve this is by splitting documents into segments and classifying them with standard methods separately. However, classification results have to be aggregated on document level again afterwards. We introduce our comparative study of different aggregation algorithms and classification techniques in the context of text classification for public administration.

joint work with Michael Trebing and Stefanie Schwaar

Melina Freitag

(Universität Potsdam)

Nicole Mücke

(TU Braunschweig)

Anke Stoll

(FDTech Chemnitz)

Stefanie Schwaar

(Fraunhofer ITWM Kaiserslautern)

Leena Chennuru Vankadara

(Universität Tübingen)

Melanie Kircheis

Kirandeep Kour

When data are high-dimensional, efficient learning algorithms must utilize the tensorial structure as much as possible. The ever-present curse of dimensionality for high dimensional data and the loss of structure when vectorizing the data motivates the use of tailored low-rank tensor methods. Also, defining a basic Machine Learning (ML) model for such data is implicit. Therefore, exploiting the tensorial structure and extracting features to find the best fitted tensorial ML model does not hold a generic vectorized approach.One of these well-known classification ML models is the Support Tensor Machine, where a non-linear boundary value problem leads to finding the best kernel approximation. Hence, dealing with these models is a two-step problem that includes finding a best-suited low-rank approximation for the best-suited kernel approximation.The classical tensor factorization methods such as tucker, and tensor-train fails to capture the right features for the state-of-the-art Dual Structure-preserving Kernel (DuSK). We have observed that these classical tensor factorizations lack very specific qualities like norm equilibration. In this work, we have tried to minimize the gap between two steps, a low-rank approximation and kernel approximation, to achieve a state-of-the-art classification model. We have introduced a new tensor factorization in tucker form, as well as a new kernel approximation function, known as Weighted Subspace Kernel. With numerical examples, we show that this approach can outperform state-of-the-art techniques.

joint work with P. Benner, S. Dolgov, M. Pfeffer and M. Stoll

Fatima Antarou Ba

Franziska Nestler

Theresa Wagner

Kernel matrices are crucial in many learning tasks and typically dense and large-scale. Depending on the dimension of the feature space even the computation of all its entries in reasonable time becomes a challenging task. For such dense matrices the cost of a matrix-vector product scales quadratically with the dimensionality, if no customized methods are applied. In basically all kernel methods, a linear system must be solved. Our approach exploits the computational power of the non-equispaced fast Fourier transform (NFFT), which is of linear complexity for fixed accuracy. The ANOVA kernel has proved to be a viable tool to group the features into smaller pieces that are then amenable to the NFFT-based summation technique. Multiple kernels based on lower-dimensional feature spaces are combined, such that kernel-vector products can be realized by this fast approximation algorithm. Based on a feature grouping approach this can be embedded into a CG solver within a learning method and we nearly reach a linear scaling. This approach enables to run learning tasks using kernel methods for large-scale data on a standard laptop computer in reasonable time without or very benign loss of accuracy. It can be embedded into methods that rely on kernel matrices or even graph Laplacians. Examples are support vector machines or graph neural networks that can then benefit from having the fast matrix-vector products available. Moreover, we make use of this method for Gaussian process regressors, where the covariance matrix is a kernel matrix. By applying our NFFT-based fast summation technique, fitting the kernel by tuning the hyperparameters, so making predictions by posterior inference can be accelerated.

joint work with Franziska Nestler and Martin Stoll

Angela Thränhardt & Martina Hentschel