Smart Tags and Enhanced Smart Tags

Today every organization creates and manages large amounts of digital data to provide better user-experience. Managing assets requires substantial amount of effort from library managers to actively tag assets.

Smart tag feature aims to reduce management effort by automatically identifying and tagging images with generic tags. This is powered by Adobe Sensei, which uses Adobe Stock as source of images and tags.

Even better, enhanced Smart tags enables library managers to automatically tag their assets with their business taxonomies.

Prerequisites

You need to integrate Adobe Experience Manager (AEM) with the Smart Content Service using Adobe I/O.

Before you can use the Smart Content Service, ensure the following to create an integration on Adobe I/O:

An Adobe ID account that has administrator privileges for the organization
The Smart Content Service is enabled for your organization.

Configuring Smart Tags for AEM

1. Verify admin access to organization’s admin console.

Login to https://adminconsole.adobe.com/ via your organization id
After successful login, you should be redirected to an organization overview page.
Verify the organization on top Right corner, to assure you are accessing the desired organization admin console. Click on the organization name, to choose a different organization that you might have access to.

Admin console-choose org.PNG

2. Verify if Smart Content Service is enabled

Via Admin console https://adminconsole.adobe.com/: In the overview link, look for “Smart Content Services”. If it appears, the service is enabled.

adminconsole1 - Copy.png

Providing access to other team members

Use Assign Developers link, to provide developer access to others. The following snapshot shows the developer access provided on Smart Content Services.

adminconsole2 - Copy.png

3. Obtain public certificate from AEM

A public certificate allows you to authenticate your profile on Adobe I/O. Follow steps on link, to create the certificate

The certificate created on AEM is valid for 10 years.

certificate info.PNG

Once the certificate expires, follow the steps on link, to recreate and configure new certificate

4. Create Adobe I/O integration (Adobe I/O console)

To use Smart Content Service APIs, create an integration in Adobe I/O to generate API Key, Technical Account Id, Organization Id, and Client Secret.

Follow steps on link, to create the integration on Adobe I/O. Append the env text(dev, stage, prod) in integration name for easy identification.

Create a new integration- 1 - Copy.PNG

Save the integration details, to update Smart Tag Cloud Service in AEM. The integration details would look like following snapshot.

5. Configure Smart Content Service

To configure the integration, use the values of Technical Account ID, Organization Id, Client Secret, Authorization Server, and API key fields from the Adobe I/O integration. Creating a Smart Tags cloud configuration allows authentication of API requests from the AEM instance.

Follow steps on link, to update the cloud service

At the back end, the AEM server authenticates your service credentials with the Adobe IO gateway before forwarding your request to the Smart Content Service.

6. Validate the configuration

The integration between AEM and Smart Content Service can be validated using “SimilaritySearch Miscellaneous Tasks”. Details can be found on link

If any of the “Configuration Check” fails, Smart Tagging or Training will not work.

Revisit Adobe I/O configuration

You can revisit/review the Smart Tag Configuration by logging to the https://console.adobe.io/. Assure that you are browsing the console for the desired organization.

Search for Assets via Smart Tags

The Smart tagged assets can be searched via Omnisearch box. The steps are available on link.

Out of the box, we cannot search assets via Filters Panel. The primary reason would be that smart tags are not stored in a central location in AEM. The smart tags associated with an asset are stored below (dam:Asset)/jcr:content/metadata/predictedTags

For better understanding of the search results visit link

Enhanced Smart tags

Compared to natural language vocabularies, tagging digital assets based on business taxonomy helps align them with a company’s business and ensures that the most relevant assets appear in searches.

For example, a car manufacturer can tag car images with model names so only relevant images appear when images of various models are searched to design a promotion campaign.

In summary, here are the main steps to use the Smart Content Service:

Onboarding
Reviewing assets and tags (taxonomy definition)
Training the Smart Content Service
Automatic tagging

Details on Training the Smart Content Service can be found at link

Custom Smart Tags are stored in the exact same manner as other predictedTags. The node name and the name property are derived from the cq:tag id used in the Smart Tag Training. Example: “namespace.tag_name1”

Fine tuning Smart Tag config:

There is an osgi config to fine tune the smart tags applied. This can be achieved by com.day.cq.dam.similaritysearch.internal.impl.SimilaritySearchServiceImpl.

Options available are:

max.tags : The maximum number of tags that are predicted. Default is 25.
min.tag.confidence: The minimum confidence score below which tags are not predicted. Default is 0.5
training.namespace: separate training from other namespaces in the instance of the service.

To modify a parameter:

Create a “install” folder inside crx-quickstart folder. Create com.day.cq.dam.similaritysearch.internal.impl.SimilaritySearchServiceImpl.cfg inside the folder. Edit the file to change the value. Example: training.namespace=some-test-namespace

Logging / Debugging

Set DEBUG level logs for com.day.cq.dam.similaritysearch

Things to remember for Enhanced Smart Tags:

Train the service multiple times to improve its ability to apply relevant tags. After each training cycle, run a tagging workflow and check whether your assets are tagged appropriately.
Assure images are fully tagged. For example, a training image might be associated with two custom tags, but the user only tags it with one. In this case, without any checks, the training would end up using the image as a negative for the missing custom tag. This would result in poor accuracy of the corresponding lightweight classifier model.
Follow the Enhanced Smart Tags Training Guidelines
First time training requires at least two set of different images and tags with sufficient no. of assets to be able to train the tags successfully. Please refer to https://helpx.adobe.com/experience-manager/6-4/assets/using/smart-tags-training-guidelines.html for more details.
To send the images for training again, the asset/metadata must be modified first. i.e. lastModified > lastTransferredForTraining

Generate report for trained tags

User can generate report from Tools > Assets > Reports > Smart Tags Training.
User can view the report which lists all the trained tags and their respective confidence (High, if greater than 75% ,Medium, if greater than 60% and lesser than 75%)

ACL set up for reports can be found at link

More Info:

Filtering/moderating tags: Not available. The Smart Content Service returns a list of asset tags. These can be removed manually using the Manage Tags screen or via an automated process. The details for moderating smart tags can be found on Managing Smart Tags
Only the 319 Jpeg Thumbnail rendition is sent to the Smart Content Service.

Error and Fix:

Q. While validating the configuration via JMX mbean (http://<cqhost>/system/console/jmx/com.day.cq.dam.similaritysearch.internal.impl%3Atype%3Dsimilaritysearch) , I receive : Failed to get access token from authorization server status: 400 response: {“error_description”:”No valid bindings were found for organization and technical account combination”,”error”:”invalid_token”}.

A. Verify that organization Id & Technical account id combination is a valid one in the cloud configuration.