Welcome to the 14 Data and AI Challenges Report!
The realm of Digital Marketing never ceases to surprise us. At the time of writing (December 2023), Scott Brinker’s Marketing ‘Technology Landscape’ has expanded from a mere 150 solutions in 2011 to now 11,038. In recent years, new questions arose like, “What is the long-term role of CDPs (Customer Data Platforms)?” or “Should a Data Warehouse-native approach serve as the organization’s data cornerstone?”
And things only seem to accelerate. Since OpenAI launched ChatGPT 3 on November 30, 2022, there has been a whirlwind of new releases and products from tech giants and startups alike. These present tons of opportunities but many challenges as well.
How do you make sense of it all? At DESelect, we love to talk to customers and leaders in the industry to understand user needs. We interviewed leaders and experts in Marketing Operations and Marketing Automation across the US and Europe. Our main goal was to uncover challenges related to data and Al, but many other challenges presented themselves as we went along.
We thank everyone who took the time to speak with us. This Data and AI Challenges Report would not be here without you!
This paper starts with our objective and methodology. Then, we share the common challenges we have identified and add some of our insights. We conclude by providing you with a glimpse of what this all means for DESelect and an afterthought.
To see our co-founders unveil the 14 Data and AI Challenges Report, you can watch our webinar on demand.
Challenge 1: Lack of integrated systems and complex data models
A recurring theme is the challenge of ensuring high quality, clean, and integrated data across various systems. This includes difficulties with consolidating data from multiple sources, handling duplicates, and integrating systems like CRM and ERP. Many of our interviewees have to deal with multiple Marketing Automation Platforms (MAPs), often the result of a decentralized organizational structure or the result of mergers and acquisitions.
As a result, there is a challenge in getting access to all data points in the customer journey, especially offline touch points.
Existing data models may be too complex or not flexible enough to meet current marketing needs, leading to the desire for more simplified, marketer friendly systems that reduce the dependence on IT and engineering teams. One interviewee reported having 750 tables in the CRM!
A typical challenge for marketers is that most databases use a relational structure, but to send out a campaign the data needs to be “flattened.” Some platforms like Salesforce Marketing Cloud offer a great advantage here by allowing marketers to flatten the tables themselves through SQL or third-party apps like DESelect Segment.
Almost everyone wants to achieve a full 360 view of the customer. There is a challenge to get data living in different systems into one place, and then correctly join the data. This can be challenging as the same contact can have different identifiers in different systems. A more pragmatic approach may be to ask “What data is really needed to solve our use cases?”
The idea of a 360-degree view has been around for as long as there have been databases. However, we do believe customers should consider what will be their main “data hub.” This will most likely end up being either a Customer Data Platform {CDP) or a Data Warehouse {DWH)- we haven’t heard arguments strong enough to have both.
Challenge 2: Effective data management and dependence on other
Challenges in managing data effectively encompass segmentation for campaigns, handling large datasets, ensuring accuracy, and dealing with issues like duplicates. Improved data governance, quality control, and accessibility are crucial for efficient marketing operations.
Dependency on upstream teams or external agencies for data updates creates bottlenecks and frustrations, especially when data quality issues arise. DESelect advocates for enabling marketers to manage data independently to enhance understanding and performance. Reliance on agency partners for campaign data ownership impedes effective campaign optimization and marketing maturity.
There is also often a lack of data enablement. A significant challenge is ensuring the integrity and availability of internal first-party data for marketing purposes. Multiple interviewees had new integration projects on their roadmap and expressed concern about a lack of transparency regarding the quality or even the nature of the data. Stakeholders often have unrealistic expectations about how difficult it is to harmonize this data on the marketing side.
Challenge 3:
Data utilization and personalization
Along similar lines, participants have expressed a difficulty in appropriately formatting data for mass personalization. This issue concerns not only their customer data, but also so-called “content data,” which ranges from bits of text to images. As one interviewee comments: “In our ideal world, we would only have one email template and a long table of content data that gets fed into it for every single send in a personalized and automated way.”
In some situations, the data is actually available, but its use for marketing purposes is restricted. For example, banks would love to tailor messages based on a customer’s transactions, but that is not legally allowed. Another example was a retailer whose physical store data and online retail data belonged to two different legal entities that were not allowed to share data.
Marketers are also noticing that consumers are becoming more hesitant to provide their data.
A related challenge is that some marketers want to take their personalization to the next level and A/B test a lot more. However, this creates the need for more content pieces and workload than they can carry.
Challenge 4: Complexity in systems and processes
There’s a clear need for more streamlined processes and better project management practices.
Many interviewees express difficulties with process inefficiencies and the complexity of marketing technology. This includes managing complex projects, coordinating across teams, adhering to deadlines, and dealing with last-minute changes or demands. There’s a clear need for more streamlined processes and better project management practices.
However, for many, this also presents one of the greatest opportunities. Ways to consolidate work, for example across brands or markets, can yield enormous efficiency gains. As one interviewee said: “When you take a step back, you realize there are more similarities than differences.”
We note that the best-in-class marketing teams set out early to build standard email templates, standard data definitions, and so on.
Challenge 5: Knowledge gaps
Many interviewees report challenges with training marketing teams on the data model and marketing technology. An organization might possess the most advanced data model and tools, but as long as people don’t know how to use them, they are useless.
The more user-friendly the tools and the less they require a non-technical person to understand the data model, the easier the onboarding process of new tools and people. Note this is a never-ending process, as there will always be people leaving and joining the team.
Challenge 6: Resource and budget constraints
Professionals are struggling with managing their time effectively across various tasks and responsibilities. Many even have to work with reduced budgets. This can lead to restraints on both people and technological capabilities. This challenge manifests in a heavy day to-day workload that impedes progress on strategic projects, as well as in the reliance on external agencies which may not always align with the companies’ processes or speed.
The challenge is exacerbated by the need to juggle multiple roles or cover for inadequacies in the system, which leads to operational inefficiencies and extends work hours. One more extreme example was a participant with only one Salesforce developer for an organization of over 14,000 employees.
Companies care about how long it takes to get a campaign out of the door, as this impacts how quickly they can respond to current events, and how many customer touchpoints they can process. Campaign throughput times can vary from 2 days to multiple weeks, depending on the complexity.
Challenge 7: Understanding AI use cases and related concerns
Many interviewees are exploring Al but haven’t implemented it yet. Interest lies in using Al for marketing insights and automation, but challenges include understanding its role in strategy and implementation complexities. Interviewees also reported that there is still a lack of clarity from vendors like Salesforce on exactly what and when the capabilities will be.
However, some organizations have formed task forces to explore Al’s industry-specific use cases. Excitement surrounds Al’s potential to solve issues and transform processes, particularly in content generation and campaign optimization. One organization mentioned using it to do sentiment analysis on reviews, so they could exclude dissatisfied customers from campaigns and to detect trends in specific aspects of their products.
Some organizations take it a step further by typically engaging an internal data science team to work with machine learning experts to create a model determining the best course of action. For example, a retailer would want to identify which customers are most likely to make a purchase in the next week and would send them a personalized offer. One customer has seen over a 60% increase in sales with this strategy. Some organizations also have ambitions to use machine learning for affinity detection, so they can personalize content.
Concerns arise regarding Al’s impact on personnel, including potential complacency, reliance, and even inaccuracy. There are also worries about Al’s “black box” nature, as marketers desire transparency in decision-making processes, especially in journey execution.
Challenge 8: Internal collaboration and scaling
Effective communication within teams and across departments is sometimes implied as a challenge, especially when discussing project management and aligning on priorities. Ensuring that teams are in sync and that there is clarity in roles and expectations could lead to more cohesive and efficient operations. Additional challenges occur when political or organizational silos prevent seamless collaboration.
In campaign planning, there are challenges regarding internal communication. These include multiple teams advocating for the campaigns for their own products to be sent first with the largest possible audience, while on a company level system must be put in place to make sure individual contacts don’t get oversaturated. With increased automation, such as ongoing journeys and overlapping segments, it’s almost impossible to foresee the impact of campaigns on individual contacts.
Expanding operations to new international markets introduces its own complexities, such as dealing with new data sources, privacy regulations, languages, and market-specific marketing tactics, implying a need for more robust global data strategies.
Challenge 9: Understanding customers and their lifecycles
Managing and understanding the customer lifecycle, particularly in terms of communication overlap and customer journey mapping, is a major challenge, indicating a need for tools that offer a comprehensive view of customer interactions.
There needs to be more clarity in aligning marketing efforts with the actual needs and preferences of users or customers. This includes effectively segmenting audiences, understanding customer journeys, and delivering personalized content.This is also related to the difficulty in tracking the marketing funnel from engagement to conversion due to disconnected data models.
Challenge 10: Lack of actionable insights and changing stakeholders' mindsets
Challenges in deriving actionable insights from marketing campaigns and activities are evident. This includes difficulties with reporting tools, accessing meaningful analytics, and understanding customer engagement and behavior.
Some interviewees report cultural and behavioral challenges in adopting a more data-driven approach within the organization. A significant challenge is shifting the mindset of the team towards more dynamic and targeted marketing strategies. Overcoming entrenched thinking and the routine of batch-and-blast email campaigns is a critical obstacle.
In most organizations, reporting on marketing performance is done through a data warehouse or data lake, with a Bl tool linked to it.
SFMC also provides basic reporting functionality and more advanced reporting functionality through Datorama. Some organizations have less advanced reporting functionalities and report through Excel.
Others also use SQL and DESelect Segment to look for trends and patterns in the data that resides in SFMC.
Challenge 11: Marketing attribution and demonstrating the value of technology
Professionals are dealing with difficulties in attributing sales and conversions accurately, as well as measuring campaign performance effectively, especially in the face of increasing privacy regulations.
Multiple interviewees also alluded to the challenge of estimating the added value their programs or investments in technology can bring, let alone communicating this effectively to other stakeholders, including their leadership teams. Even with all the relevant data gathered in one place, the effectiveness of marketing activities is difficult to report on.
Another challenge is the frequency of purchase. Car purchases for example only happen once every 5 years or so, and as a result many marketing touch-points in the years leading up to a purchase could have contributed to keeping the prospect warm and the purchase happening.
We have developed multiple models based on efficiency gains, hard cost savings, revenue drivers, engagement improvement, and risk avoidance. Feel free to reach out if you’d like to learn more.
Challenge 12: Cross-channel marketing and "traffic control"
Many participants highlighted the challenge of managing audiences and traffic across multiple platforms and channels.
The difficulty lies in ensuring that customers do not receive conflicting or overlapping messages from different brands or channels within the organization. Often there’s a concern about over-saturating customers with too many messages, especially if they’re affiliated with multiple segments within the organization.
This challenge is exacerbated when there are multiple teams. In some organizations, this leads to extraordinarily expensive planning meetings or even causes the introduction of internal roles that act as liaisons between brands or marketing teams.
In the case of different brands that belong to the same group but that are considered standalone by the consumer, this is often less of a concern. We’d say though that you’d still want to spread messages to avoid marketing fatigue.
Challenge 13: New systems, legacy systems, and knowledge transfer
The need for training and skill development is evident. One interviewee reported having to learn Marketing Cloud “on the fly.” This suggests a broader requirement for ongoing education and support to keep teams up to-date with complex systems and practices. As personnel change, essential knowledge about these systems can be lost, making it difficult to understand and integrate new tools.
There’s a need to understand the data model well, particularly when doing segmentation.
Interviewees expressed challenges with navigating complex software tools and platforms, especially when inheriting systems others set up. The need to understand and fix existing configurations without adequate documentation or previous knowledge is a notable source of frustration.
Challenge 14: The Economy
The economic climate is still uncertain.
A combination of high interest rates and high inflation are impacting customers’ willingness/ability to spend, especially for purchases that are typically financed through loans, such as cars.
Marketing budgets are also under pressure, leading to marketing organizations critically evaluating the tools they use, and in some cases completely blocking any new vendors.
If you feel like things are tougher than a few years ago, you’re not alone!
Summary Data and AI Challenges Report
As businesses increasingly rely on data-driven decision-making and automation, understanding the current trends and emerging innovations in Data and Al becomes imperative for staying competitive in the dynamic marketing landscape. Through a meticulous analysis of industry reports, case studies, and expert insights, we hope this paper has provided a comprehensive overview of the key trends shaping the intersection of Data and Al in marketing:
- Without proper training, adapting many different technologies and software can hinder productivity. There is an important learning curve period that must be considered before expecting positive results from new
- It is key to strike a balance between collaboration and autonomy. Communication between different departments can render great outcomes, however it is crucial to keep in mind how it might prolong the timeline of a project
- Thorough internal documentation is one of the most important practices for quickly evolving enterprises. The adoption period of new technologies, as well as change in management and personel, can be minimized with the use of informative and comprehensive records.
The insights extrapolated from the interviews can be utilized as a guiding light to navigate such pressing issues in the rapidly evolving current market. To watch our co-founders walk through the report, watch our webinar on demand.
To learn how DESelect can help you create premium customer experiences that drive revenue, brand loyalty, and ROI, we invite you to schedule a meeting today.
