Either in stochastic optimization or statistical learning, the goal is to find a solution minimizing on average a certain population risk. Classical methodologies assume the collected data is clean and light-tailed. Beyond that setting, approaches based on the sample average are severely deteriorated with even a single outlier.  Algorithmic "robust'' solutions of these problems aim in designing estimators that can be reliable under corrupted or heavy-tailed samples while still being computationally tractable. On the side, one workhorse algorithm in machine learning and stochastic optimization is the stochastic gradient descent (SGD). SGD is well known to suffer from instability if the step-size is not tuned properly. Designing efficient robust SGD-type methods that are adaptive to the step-size tuning is still a major challenge in the area, either theoretically or practically.  We introduce the audience to our research projects, namely, algorithmic robust statistics and adaptive SGD-type methods. While traditionally from different fields, it is now a common trend to unify techniques from both Statistics/Machine Learning and Optimization communities. Both research programs have an unifying theme: (1) to weaken traditional assumptions on data, e.g., contamination, heavy-tails or non-uniformly bounded variance,  (2) to achieve statistical and optimization optimality while (3) designing practical methods (i.e. stable and adaptive) with theoretical guarantees.