Feature Extraction

Feature extraction is the step that turns raw data into numbers a machine learning model can use. Models cannot read a keyword, a page or a product photo directly, so each item is converted into a list of numerical features (a vector) that captures what matters about it. The choice of features largely decides what the model is able to notice.

How it works

The method depends on the data type:

• Text. After tokenisation and cleaning, text is vectorised with counts (bag of words), weights (TF-IDF) or dense embeddings from models such as BERT. Counts and weights give long, sparse vectors with one column per word; embeddings give shorter, dense vectors that capture meaning.
• Images. Colour histograms, dominant colours, or embeddings from a vision model.
• Tabular data. Derived features, such as recency, frequency and monetary value calculated from raw orders.

Feature extraction is usually followed by dimensionality reduction, such as PCA or UMAP, when the vectors are too large to cluster or plot well.

How it’s used in SEO and marketing

• Entity mapping. In the Semantic Keyword Research course, entity names are vectorised with TF-IDF, clustered with K-means, then reduced to two dimensions with PCA for a visual map.
• Topic modelling. BERTopic embeds documents, reduces them with UMAP, clusters them with HDBSCAN, then uses class-based TF-IDF to describe each topic.
• Image clustering. MLforSEO’s colour clustering practical extracts colour features from product images before grouping them.
• Classification. Features such as query length or question words feed intent and query-type classifiers.

Example

Take a list of finance entities: “personal loan”, “car loan”, “credit card”, “credit score”, “crowdfunding”. TF-IDF gives common words such as “loan” and “credit” less weight because they appear across many entity names, and gives a distinctive word like “crowdfunding” more. The resulting vectors let a clustering algorithm group the loan entities together while keeping crowdfunding apart.

Limitations and things to watch

• Poor features cannot be rescued by a better model.
• Sparse count-based features ignore meaning; embeddings capture meaning but are harder to interpret.
• Apply exactly the same pre-processing and vectoriser to new data as to the training data.
• Avoid leakage: never build features from information the model would not have at prediction time.

Learn it on MLforSEO

• Topic Modelling for SEO with BERTopic
• Different ways to Map Keywords to Topics
• Image Clustering by Color with Three Machine Learning Approaches
• Semantic AI-powered SEO Keyword Research course
• Introduction to Machine Learning for SEO course