The effect of an outlier depends upon the choice of the algorithm (logistic regression is more sensitive to outliers than random forest) and also upon the use requirement of metric tolerances (for e.g. how good should your f1score be).
Most of the outlier detection methods are univariate i.e. they remove outliers from a single variable at a time by either using an IQR method or using a z-score method and so on. However, let’s consider a simple example, to understand a limitation of these methods. For transactions data analysis (to identify a fraud one let’s say), say a customer buys some appliances, with transaction amounts much higher than normal compared to other customers. It’s fine, he/she might be rich. However, if that customer has a very low age (probably not even 18), then it’s a question mark!
Solution: Remove outliers on mathematical combinations of features (reduced from PCA for e.g.)