Deep Dive: Technical Challenges Unique to AdTech and MarTech

For three years I served as CTO at a digital marketing agency with $100M+ in revenue. We built several AdTech products from scratch and solved problems that are rarely encountered in other industries. Not because they're academically harder, but because the cost of a mistake is directly measured in client money — and it becomes visible within hours, not sprints.

In this edition, I'll break down the specific technical challenges we faced and the approaches that worked.

1. End-to-End Analytics: When the Data Pipeline Is the Product

In most SaaS companies, the data pipeline is supporting infrastructure for the product. In MarTech, the pipeline itself becomes the product.

We built an ETL system that automatically pulled data from ad platforms (Google Ads, Yandex Direct, Facebook Ads, myTarget, and others), web analytics systems, call tracking services, and clients' CRM systems. Then came transformation, cleaning, and merging. The output — dashboards showing the full picture from click to sale.

Sounds like standard ETL. In practice, every data source lives by its own rules. Ad platforms have different attribution models, different data granularity, different time zones. Even costs are reported differently — some platforms return amounts with VAT, others without, and this must be accounted for when consolidating data. A client's CRM might deliver data with a day-long delay. Call tracking ties calls to sessions using its own logic. And all of this needs to come together into a single coherent picture.

A separate pain point — stability. Ad platforms change APIs, break backward compatibility, introduce rate limits. If the pipeline goes down overnight and yesterday's data doesn't load, the account manager on the client side makes decisions based on incomplete data in the morning. We arrived at a point where monitoring and reconciliation jobs (automated verification of loaded data against sources) aren't just a nice-to-have — they're an integral part of the system.

2. Feed Management: Real Estate Developer Specifics

When people talk about data feed management in AdTech, they usually mean product feeds for e-commerce. Our primary use case was different — feeds for real estate platforms: Yandex Real Estate, Avito, CIAN.

Real estate is more complex than retail for several reasons. The object isn't a "product in a warehouse" but an apartment in a building under construction, with a status that changes (available, reserved, sold), along with price, floor plan, and visualization. Each platform has its own feed format, its own image requirements, its own set of mandatory fields. CIAN wants one structure, Avito wants another, Yandex Real Estate wants a third.

Plus the business logic: which units to show, which to hide, how to manage priorities depending on the sales stage. A developer doesn't want to advertise apartments that are already sold, but data about this arrives from their CRM with a delay.

We built a system with adapters for each platform and a reconciliation mechanism that caught discrepancies between the feed and the actual state of units. Essentially the same pattern as in end-to-end analytics: abstraction over unstable external systems, aggressive monitoring, minimizing consequences when changes occur.

3. Econometrics and Marketing Mix Modeling

The classic question for any marketing media planning: "Which channel actually generates revenue, and which one just claims credit for conversions?" Standard attribution from Google Analytics doesn't answer this — it shows which channel was last in the chain, not which one actually influenced the purchase decision.

We implemented Marketing Mix Modeling based on an ML model — essentially, econometric analysis of each advertising channel's impact on business outcomes. Key components:

• AdStock effect. Advertising in most cases doesn't work instantly. As a rule, the more expensive the product, the longer the sales cycle. A person sees an ad today but buys a week later. You need to model the decay of advertising impact over time, and the decay coefficient is different for each channel.

• Saturation limit. Every channel has a point after which additional investment stops delivering proportional returns. Pouring more money into a channel past this point means burning budget. The model allowed us to find the optimal budget distribution across channels.

• ROAS analysis. The output isn't abstract "insights" but concrete numbers: how much revenue each ruble invested in a specific channel generates, accounting for the delayed effect and saturation.

Technically this was implemented in Python (NumPy, Pandas, Sklearn, FB Prophet), with fairly heavy computation on clients' historical data. The main engineering challenge wasn't the model itself but the data. For MMM to work, you need clean, complete, and properly labeled data on channel spending and business results over a long period. That same ETL system from the first section.

4. Data-Driven Attribution: Markov Chains and Shapley Value Instead of Last Click

In parallel with MMM, we worked on data-driven attribution — the problem of distributing "credit" for a conversion among touchpoints in the user journey.

Two approaches we applied:

• Markov Chains. We model the user journey as a graph of transitions between channels. For each channel, we calculate the removal effect — how much total conversions would drop if that channel were removed from the chain. Channels with a high removal effect receive more "value" for the conversion. This approach works well when there's enough data to build a statistically significant transition graph.

• Shapley Value. A game theory approach — each channel is treated as a "player in a coalition," and its contribution is evaluated by the additional value it brings to every possible combination of channels. Mathematically elegant, but computationally expensive — the number of coalitions grows exponentially with the number of channels.

Both approaches showed a picture that differed significantly from standard last-click in Google Analytics. It often turned out that upper-funnel channels (display advertising, OLV) that looked useless under last-click were actually initiating chains that led to conversions.

The engineering challenge here is scale. Large clients have millions of sessions and thousands of unique paths. A naive Shapley Value implementation hits the computational ceiling, so approximations must be applied. Plus all the same problems with user identification: to build a user journey, you first need to collect all touchpoints of one person into a single chain, which is a separate challenge at the intersection of data, privacy, and browser technical limitations.

5. Audience Clustering and Lookalike Targeting

One of the tasks where ML delivered a real, measurable business result was audience clustering.

The setup: we collect data from a DMP (Data Management Platform) about users who interacted with the client's ads. Each user has a set of characteristics: demographics, interests, on-site behavior, purchase history. Then ML algorithms group users into clusters based on similar traits.

The most valuable part isn't the clusters themselves but identifying dominant traits in the converting audience. For example: "your best buyers are men aged 30-40, interested in cars, who visited the site 3+ times in the last month." This provided concrete recommendations for the creative team — which audiences to tailor ad messaging and visuals for.

The next step — exporting these segments to Facebook Ads and Google Ads to create lookalike audiences. The platforms take your segment and find similar users you haven't reached yet. In practice, conversion rates for such audiences were noticeably higher than for standard targeting.

The technical side: Python for the ML part (clustering, feature importance), DMP integration via pixel for data collection, integration with ad platforms for segment export. The main engineering problem isn't the algorithms (standard approaches like k-means and random forest work fine) but the quality of input data and pipeline automation to ensure segments are updated regularly, not as a one-time exercise.

6. A/B Tests and Multi-Armed Bandit Algorithms

Standard A/B testing assumes you can randomly split traffic and independently measure results. In advertising, this breaks down quickly.

Ad campaigns have budget constraints, audience overlap, and temporal effects that prevent them from being treated as independent. If you shift budget from campaign A to campaign B for a test, you're not just testing B — you're also reducing competition in the auctions where campaign A participated, which changes the results of both campaigns.

We researched and published materials on using multi-armed bandit algorithms as an alternative to classic A/B testing in the advertising context. "Bandits" dynamically adapt budget distribution based on observed results — instead of splitting traffic 50/50 and waiting for statistical significance, the algorithm gradually redistributes budget toward the better-performing variant.

Pros: fewer losses on the losing variant, faster adaptation. Cons: slower convergence to a statistically reliable conclusion, harder to interpret results, more difficult to implement correctly.

Geo-testing turned out to be the most reliable approach for measuring the incremental effect of ad spend. Take similar geographic regions, run a campaign in some and not in others, compare results. Not trendy, but it produces numbers you can trust.

7. Recommendation Engines for E-Commerce

A separate direction — ML-based recommendation systems for the agency's e-commerce clients. The task is classic: show the user products they're most likely to buy, based on their behavior and the behavior of similar users.

What's interesting here isn't the model itself (collaborative filtering, content-based approaches — all standard) but the context of application. In an agency, the recommendation engine isn't the company's internal product but a service that needs to be deployed and integrated with the client's infrastructure. Each client has their own stack, their own data, their own understanding of what counts as a "product" and a "conversion."

This creates a need for architecture that's flexible enough to adapt to different clients but doesn't turn into a collection of one-off custom solutions. We arrived at a modular structure: a shared core with pluggable data adapters and model configurations for each specific client.

What This All Means If You're Building a Product in AdTech/MarTech

The seven sections above cover seven different engineering challenges, each with its own pitfalls. But common patterns emerge:

• Data is the foundation of everything. Neither MMM, nor clustering, nor attribution work without clean, complete, and timely data. Investment in ETL infrastructure pays off across all other areas.

• External dependencies will break. Google, Meta, Yandex change APIs on their own schedule. You need to build with instability in mind — adapters, abstractions, monitoring.

• Domain expertise matters. Someone who has already gone through econometric modeling at scale or built data-driven attribution will save months of trial and error. In this vertical, the gap between "I understand ML" and "I can apply ML to advertising data" is enormous.

---

My name is Anton Golosnichenko, I'm a fractional CTO and technical consultant. For over 3 years I led the technology department at a digital marketing agency with $100M+ in revenue, where we built everything described above. Now I help AdTech, MarTech, TravelTech, and SaaS startups scale their engineering teams and technical infrastructure. If your startup is facing technical growing pains, I'd be happy to connect. Book a free consultation: https://golosnichenko.com