Mighty Digital's Singer Tap for Apple Search Ads
Share
As with any advertising platform, it is important that you regularly test the performance of your Apple Search Ads campaigns to ensure that your money is being spent wisely. While Apple Search Ads will improve the visibility of your application by featuring it prominently in App Store search results, your advertising budget is being wasted if your ads are not reaching the right viewers.
Unfortunately, Apple Search Ads’ built-in analytics tools are somewhat limited for this purpose. To accurately gauge the effectiveness of your advertising campaigns, you need access to the raw data that is collected whenever a user views, clicks, or scrolls past your ads.
Singer provides an open-source ETL platform that allows you to extract data from a variety of sources and transform it into a consistent format for storage and future use. We wanted to use Singer to extract and transform data from Apple Search Ads so that we could process it ourselves and gain deeper insight into the behavior of those seeing our ads. This enabled us to target them better and see better returns on our advertising investments.
That brings us to why we’re here. There was no existing integration (known as a tap) between Singer and Apple Search Ads — so we’ve created one, and we’re open-sourcing it so that you can use it too!
Apple Search Ads and the importance of search analytics
Introduced in 2021, Apple Search Ads has become an important means to reach customers. New privacy standards like GDPR and CCPA and improved privacy controls introduced since iOS 14.5 require new approaches to outreach. Apple Search Ads provides a simple and direct way to reach consenting users from within the App store — presenting them with your advertisement while they are engaged and actively looking to install something new — without having to target them with profiling data from their internet habits.
Of course, some advertisements are more effective than others. Simply firing ads at users’ screens is not enough; the process should ideally generate insights so that campaigns can be updated to reflect emerging trends and expectations. Apple Search Ads provides several types of data for this purpose: campaign, ad, and keyword level reports are available so that you can see where and when your ad is appearing and which audiences are responding to it.
While Apple Search Ads does provide a dashboard and reporting tools for monitoring your ad campaigns, it is often preferable to take full control of your data. Storing it on your own infrastructure makes it flexible and ready for any analytics you wish to run on it without limitations.
The Apple Search Ads API provides a programmatic way to retrieve data from the platform so that you can take ownership of your data. To do so, the data must be extracted and formatted and then stored for later use — often in a data lake or data warehouse.
What is a Singer ETL?
Singer.io is an open-source tool JSON-based data shifting ETL (Extract, Transform, Load) pipeline. Singer is one such tool that Mighty Digital has been leveraging with great results. Singer takes data from a source (the tap) and loads it into a destination (the target). Data is converted into the widely-adopted JSON format, which is easy to process, convert, and store.
We use Singer because it makes our jobs easier. Usually, to extract data from a new source, we’d have to spend hours reading often indecipherable API documentation, figure out how to get the data we need, and then spend time coding an integration to extract and format the data. Then, we’d have to debug it. This process is an inefficient use of both time and coffee.
Singer taps and targets
With Singer, we can take taps written by the community and start consuming data instantly — sending it straight to supported targets like Redshift or BigQuery, or even into simple CSV files. Singer includes tools for handling state changes so that our data pipelines can leave off from the point where they got stuck or crashed, ensuring no data is lost.
Unfortunately, when we wanted to integrate Apple Search Ads into one of our clients’ Singer data pipelines, we hit a snag: there was no publicly available Singer tap for Apple Search Ads!
Introducing our Singer tap for Apple Search ads
At Mighty Digital, we’ve been beneficiaries of the multitude of open source data tools that have been made available as big data has grown as a concept and as an industry. We saw this missing tool as our chance to give back — so, with no more fanfare, here’s our Singer tap for Apple Search Ads.
Download our Apple Search Ads Singer Tap
Our Apple Search Ads Singer tap supports Singer’s discovery mode and offers both raw and flat objects from the available data streams. Streams providing flat objects can be sent straight to SQL database targets such as pipelinewise-target-postgres or to flat-file targets like target-csv, while streams providing raw objects can be sent to unstructured targets such as target-json.
We won’t dive into how to install it right here — visit the repository to see the integration and our thorough, up-to-date installation instructions and documentation for the latest version.
How we built it
Our tap follows a basic design, with our initial goal being to better understand the Singer specification and figure out how hard it is to implement taps for future projects. The required API endpoints were selected based on our current usage needs, and mapped to both flat and unstructured Singer streams.
Each stream has a dedicated specification, built manually using JSON, following the data structure in use in our existing pipelines. Singer converts API responses into stream objects for each endpoint, mapping flat fields and merging nested fields as required to match the final required structure.
Once objects are processed, they are printed to stdout using singer-python, which handles the Singer specification annotations and output formatting. singer-python is also used for configuration parsing and control flow, performs some basic validation, and is used to generate the final Singer catalog.
Future plans
Currently, the configuration is handled by editing the config.json file, while stream selection is done via the catalog.json file, which is created during the discovery process.
We plan to extend the configuration to handle the selection of current streams rather than using predefined values. We’ll also be continuing to add API endpoints and corresponding streams and allow users to select the object model to use: nested JSON objects, flat objects, or normalized representations of data.
If you have any suggestions or improvements, please let us know (or submit a PR to the GitHub repository)!
Apple Search Ads Singer tap — what is it good for?
Now, we have full access to and ownership of, our Apple Search Ads data. We can store, analyze, and build insights spanning the entire history of our data from across a variety of sources.
We‘ve been combining our Apple Search Ads data with data from other advertising sources and in-app user interactions to build a cost of acquisition for each of our clients’ users. The Apple Search Ads built-in reporting tools don’t allow for this data to be reformatted and mixed with other data, which limits its usefulness.
We’re hoping that the Singer community will benefit from this tool and be able to get more value from their own Apple Search Ads data using it.
Made by Mighty Digital with ❤️
We’ve learned a lot from the data community — we wouldn’t be where we are today without them. Open source benefits everyone, and Singer’s open-source ETL platform is a well-built, robust, and flexible foundation for building future data tools.
At Mighty Digital, we’re hoping that our Apple Search Ads Singer tap is the first of many tools that we will be able to open source and provide. We’re giving back, and we want to keep giving back.
If you’re a data nerd, come and join us on Twitter if you want to talk shop. If you’re a business looking to make the most of your data, get in touch to find out how we can make sure your data is being used to its full potential.
