Skip to main content

01 - Newsletter Sign-up

Introduction​

This page introduces the main elements of the form definition schemas required for a simple newsletter sign-up. It uses the corresponding data submission payload to help explain the schemas, recognizing there is an obvious relationship between the data needing to be captured and the definition of the form used to capture that data.

Form Definition Schema​

With a JSON-first mindset, we’ll define our form using JSON Schema (JSON Schema ), so a simple newsletter sign-up form will look initially like:

Initial Newsletter Sign-up Schema - https://digitalmarketing.schemas.essity.com/newsletter-sign-up.schema.json
{
  "$schema":"https://json-schema.org/draft/2020-12/schema",
  "$id":"https://digitalmarketing.schemas.essity.com/newsletter-sign-up.schema.json",
  "title":"Newsletter Sign Up",
  "description":"Newsletter sign up request",
  "type":"object",
  "properties":{
    "user":{
      "description":"User's email address to be added to newsletter mailing list",
      "$ref":"https://digitalmarketing.schemas.essity.com/user-profile.schema.json"
    },
    "newsletterSubscription":{
      "type":"boolean"
    }
  }
}

What we see here, is that we are capturing personal data related to the user profile, so we need to refer to the user profile schema

Initial User Profile Schema - https://digitalmarketing.schemas.essity.com/user-profile.schema.json
{
  "$id":"https://digitalmarketing.schemas.essity.com/user-profile.schema.json",
  "$schema":"https://json-schema.org/draft/2020-12/schema",
  "description":"A representation of a user profile",
  "type":"object",
  "required":["firstName","email"],
  "properties":{
    "firstName":{
      "type":"string"
    },
    "lastName":{
      "type":"string"
    },
    "email":{
      "type":"string",
      "format":"email"
    },
    "birthDay":{
      "type":"integer"
    },
    "birthMonth":{
      "type":"integer"
    },
    "birthYear":{
      "type":"integer"
    },
    "country":{
      "type":"string"
    },
    "language":{
      "type":"string"
    }
  }
}

To complete a basic user profile, we require only the “firstName” and “email” property. However, this is not enough from a business perspective, as we also require the consents from the user in order to send them communications such as newsletters, handle their personal data etc..

So we define user consent objects for each relevant consent type using

Initial User Consent Schema - https://digitalmarketing.schemas.essity.com/user-consents.schema.json
{
  "$schema":"https://json-schema.org/draft/2020-12/schema",
  "$id":"https://digitalmarketing.schemas.essity.com/user-consents.schema.json",
  "description":"A representation of user consents",
  "type":"object",
  "required":["privacyPolicy","termsAndConditions"],
  "properties":{
    "ageConsent":{
      "description":"User's confirmation of being over 18 years of age",
      "$ref":"https://digitalmarketing.schemas.essity.com/user-consent-object.schema.json"
    },
    "privacyPolicy":{
      "description":"User's acceptance of privacy policy",
      "$ref":"https://digitalmarketing.schemas.essity.com/user-consent-object.schema.json"
    },
    "termsAndConditions":{
      "description":"User's acceptance of terms and conditions",
      "$ref":"https://digitalmarketing.schemas.essity.com/user-consent-object.schema.json"
    },
    "marketingCommunications":{
      "description":"User's acceptance of marketing communications",
      "$ref":"https://digitalmarketing.schemas.essity.com/user-consent-object.schema.json"
    }
  }
}
Initial User Consent object schema - https://digitalmarketing.schemas.essity.com/user-consent-object.schema.json
{
  "$schema":"https://json-schema.org/draft/2020-12/schema",
  "$id":"https://digitalmarketing.schemas.essity.com/user-consent-object.schema.json",
  "description":"A representation of user consent object",
  "type":"object",
  "required":["isConsentGranted","lastUpdated"],
  "properties":{
    "isConsentGranted":{
      "type":"boolean"
    },
    "lastUpdated":{
      "type":"dateTime"
    }
}

so now our schema for newsletter sign-up has email, newsletter subscription flag, and consents:

Improved Newsletter Sign-up Schema - https://digitalmarketing.schemas.essity.com/newsletter-sign-up.schema.json
{
  "$schema":"https://json-schema.org/draft/2020-12/schema",
  "$id":"https://digitalmarketing.schemas.essity.com/newsletter-sign-up.schema.json",
  "title":"Newsletter Sign Up",
  "description":"Newsletter sign up request",
  "type":"object",
  "required":["user","newsletterSubscription","newsletterSubscribed","consents"],
  "properties":{
    "user":{
      "description":"User's email address to be added to newsletter mailing list",
      "$ref":"https://digitalmarketing.schemas.essity.com/user-profile.schema.json"
    },
    "newsletterSubscription":{
      "type":"boolean"
    },
    "newsletterSubscribed":{
      "type":"datetime"
    }
    "consents":{
      "description":"User's consents for newsletter mailing list",
      "$ref":"https://digitalmarketing.schemas.essity.com/user-consents.schema.json"
    },
    "version":{
      "type":"string"
    }
  }
}

Data Payload​

Knowing what we need to supply, now suggests our data payload should look something like:

Data Payload
{
  "user":{
    "firstName":"John"
    "email":"john.smith@example.com"
  },
  "newsletterSubscription":true,
  "newsletterSubscribed":"2024-10-28 13:47:52Z",
  "consents":{
    "privacyPolicy":{
      "isConsentGranted":true,
      "lastUpdated":"2024-10-28 13:47:52Z"
    },
    "termsAndConditions":{
      "isConsentGranted":true,
      "lastUpdated":"2024-10-28 13:47:52Z"
    },
    "marketingCommunications":{
      "isConsentGranted":true,
      "lastUpdated":"2024-10-28 13:47:52Z"
    }
  },
  "version":"2024-06-01"
}

Metadata​

However, we still miss the metadata supporting the data payload itself. So we need to extend our schema further to include the metadata needed.

This is slightly more complex, because we have several parent-child relationships to consider.

Form Metadata Definition Schema Relationships

First the form submission needs to include

  • the specific local campaign - often a child of a larger regional brand campaign
  • the local channel used in the campaign - it can be one of several such as brand website, social media etc. per country

So to reflect there we rely on several schemas:

Form Submission:

Initial Form Metadata - https://digitalmarketing.schemas.essity.com/form-submission-metadata.schema.json
{
  "$schema":"https://json-schema.org/draft/2020-12/schema",
  "$id":"https://digitalmarketing.schemas.essity.com/form-submission-metadata.schema.json",
  "title":"Form metadata schema",
  "description":"Form metadata",
  "type":"object",
  "required":["campaignMetadata","localMarketingChannelMetadata"],
  "properties":{
    "localCampaignMetadata":{
      "description":"Local Market campaign metadata",
      "$ref":"https://digitalmarketing.schemas.essity.com/local-market-campaign.schema.json"
    },
    "localMarketingChannelMetadata":{
      "description":"Local marketing channel metadata",
      "$ref":"https://digitalmarketing.schemas.essity.com/local-marketing-channel.schema.json"
    }
  }
}

Campaign Metadata:

(child) Local market campaign metadata schema - https://digitalmarketing.schemas.essity.com/local-market-campaign.schema.json
{
  "$schema":"https://json-schema.org/draft/2020-12/schema",
  "$id":"https://digitalmarketing.schemas.essity.com/local-market-campaign.schema.json",
  "title":"Local market campaign metadata schema",
  "description":"Local market campaign metadata schema",
  "type":"object",
  "required":["brandId","name"],
  "properties":{
    "localMarketCampaignId":{
      "type":"string"
    },
    "startDate":{
      "type":"dateTime"
    },
    "endDate":{
      "type":"dateTime"
    }
}
(parent) Marketing campaign metadata schema - https://digitalmarketing.schemas.essity.com/marketing-campaign.schema.json
{
  "$schema":"https://json-schema.org/draft/2020-12/schema",
  "$id":"https://digitalmarketing.schemas.essity.com/marketing-campaign.schema.json",
  "title":"Marketing campaign metadata schema",
  "description":"Marketing campaign metadata schema",
  "type":"object",
  "required":["campaignId","title"],
  "properties":{
    "campaignId":{
      "type":"string"
    },
    "brand":{
      "description":"Brand",
      "$ref":"https://digitalmarketing.schemas.essity.com/brand.schema.json"
    },
    "classification":{
      "type":"string"
    },
    "localMarketingCampaigns":{
      "type":"array",
      "items":{
        "$ref":"https://digitalmarketing.schemas.essity.com/local-market-campaign.schema.json"
      }
    }
  }
}

Channel Metadata:

(child) Local marketing channel metadata schema - https://digitalmarketing.schemas.essity.com/local-marketing-channel.schema.json
{
  "$schema":"https://json-schema.org/draft/2020-12/schema",
  "$id":"https://digitalmarketing.schemas.essity.com/local-marketing-channel.schema.json",
  "title":"Local marketing channel metadata schema",
  "description":"Local marketing channel metadata schema",
  "type":"object",
  "required":["channelId"],
  "properties":{
    "channelId":{
      "type":"string"
    },
    "channelType":{
      "$ref":"https://digitalmarketing.schemas.essity.com/marketing-channel-type.schema.json"
    },
    "name":{
      "type":"string"
    },
    "description":{
      "type":"string"
    },
    "referralUrl":{
      "type":"string"
    },
    "countryCode":{
      "type":"string"
    },
    "languageCode":{
      "type":"string"
    }
}
(parent) Marketing channel type metadata schema - https://digitalmarketing.schemas.essity.com/marketing-channel-type.schema.json
{
  "$schema":"https://json-schema.org/draft/2020-12/schema",
  "$id":"https://digitalmarketing.schemas.essity.com/marketing-channel-type.schema.json",
  "title":"Marketing channel type metadata schema",
  "description":"Marketing channel type metadata schema",
  "type":"object",
  "required":["channelId","locales"],
  "properties":{
    "channelTypeId":{
      "type":"string"
    },
    "name":{
      "type":"string"
    },
    "description":{
      "type":"string"
    },
    "localMarketingChannels":{
      "type":"array",
      "items":{
        "$ref":"https://digitalmarketing.schemas.essity.com/local-marketing-channel.schema.json"
      }
    }
}

Brand Metadata:

Marketing brand metadata schema - https://digitalmarketing.schemas.essity.com/brand.schema.json
{
  "$schema":"https://json-schema.org/draft/2020-12/schema",
  "$id":"https://digitalmarketing.schemas.essity.com/brand.schema.json",
  "title":"Marketing brand metadata schema",
  "description":"Marketing brand metadata schema",
  "type":"object",
  "required":["brandId","name"],
  "properties":{
    "brandId":{
      "type":"string"
    },
    "name":{
      "type":"string"
    },
    "description":{
      "type":"string"
    }
}

Metadata and Data Payload​

With the metadata definition also defined, we can now expect our form submission payload to look more like:

Form submission payload
{
  "metadata":{
    "localCampaign":{
      "localMarketCampaignId":"ABC123-GB",
      "campaignId":{
        "channelTypeId":"ABC123",
        "brand":"sample",
      },
      "startDate":"2024-10-01 00:00:00Z",
      "endDate":"2024-10-31 23:59:59Z"      
    },
    "localChannel":{
      "channelId":"www12345",
      "channelType":{
        "channelTypeId":"www"
      },
      "name":"www.sample.co.uk",
      "description":"GB Brand site for samples"
      "referralUrl":"https://www.sample.co.uk/request/",
      "countryCode":"gb",
      "languageCode":"en"    
    }
  },
  "user":{
    "firstName":"John"
    "email":"john.smith@example.com"
  },
  "newsletterSubscription":true,
  "newsletterSubscribed":"2024-10-28 13:47:52Z",
  "consents":{
    "privacyPolicy":{
      "isConsentGranted":true,
      "lastUpdated":"2024-10-28 13:47:52Z"
    },
    "termsAndConditions":{
      "isConsentGranted":true,
      "lastUpdated":"2024-10-28 13:47:52Z"
    },
    "marketingCommunications":{
      "isConsentGranted":true,
      "lastUpdated":"2024-10-28 13:47:52Z"
    }
  }
}

This now looks close to what is required…

Plus Form Instance​

… but we still lack form instance details. Imagine we have our newsletter sign-up form on every page of the website, but we want to uniquely identify the specific instance of a form, on a particular page. We could use a combination of campaign and channel (page), but these are not explicit enough by themselves.

So to address this, we extend our schema, with specific “instance” details which uniquely define the form instance on that particular page

Updated Form Metadata - https://digitalmarketing.schemas.essity.com/form-submission-metadata.schema.json
{
  "$schema":"https://json-schema.org/draft/2020-12/schema",
  "$id":"https://digitalmarketing.schemas.essity.com/form-submission-metadata.schema.json",
  "title":"Form metadata schema",
  "description":"Form metadata",
  "type":"object",
  "required":["campaignMetadata","brand"],
  "properties":{
    "localCampaignMetadata":{
      "description":"Local Market campaign metadata",
      "$ref":"https://digitalmarketing.schemas.essity.com/local-market-campaign.schema.json"
    },
    "localMarketingChannelMetadata":{
      "description":"Local marketing channel metadata",
      "$ref":"https://digitalmarketing.schemas.essity.com/local-marketing-channel.schema.json"
    },
    "formInstanceMetadata":{
      "description":"Specific form instance metadata",
      "$ref":"https://digitalmarketing.schemas.essity.com/form-instance.schema.json"
    }
  }
}
Specific form instance metadata - https://digitalmarketing.schemas.essity.com/form-instance.schema.json
{
  "$schema":"https://json-schema.org/draft/2020-12/schema",
  "$id":"https://digitalmarketing.schemas.essity.com/form-instance.schema.json",
  "title":"Form instance metadata schema",
  "description":"Form instance metadata schema",
  "type":"object",
  "required":["id"],
  "properties":{
    "id":{
      "type":"string"
    },
    "createdBy":{
      "type":"string"
    },
    "createdDate":{
      "type":"datetime"
    },
    "renderedDate":{
      "type":"datetime"
    },
    "referralUrl":{
      "type":"string"
    }
}

Updating our diagram we now have:

Form Metadata Definition Schema Relationships

Our Form metadata consists of

  • brand information
  • campaign information
  • channel information
  • form instance

which together with the actual user entries, determines the content of our submitted data payload.

So after defining the form instance, together with the (minimal in this case) user entries, we now have our complete payload, to capture our simple newsletter sign-up:

Our complete payload
{
  "metadata":{
    "localCampaign":{
      "localMarketCampaignId":"ABC123-GB",
      "campaignId":{
        "channelTypeId":"ABC123",
        "brand":"sample",
        "classification":"personal"
      },
      "startDate":"2024-10-01 00:00:00Z",
      "endDate":"2024-10-31 23:59:59Z"      
    },
    "localChannel":{
      "channelId":"www12345",
      "channelType":{
        "channelTypeId":"www"
      },
      "name":"www.sample.co.uk",
      "description":"GB Brand site for samples"
      "referralUrl":"https://www.sample.co.uk/request/",
      "countryCode":"gb",
      "languageCode":"en"    
    },
    "formInstance":{
      "id":"1ed0a66d-cf21-4be7-8e77-9a187acd838f",
      "createdBy":"dehjosm",
      "createdDate":"2024-09-30 09:25:01Z",
      "renderedDate":"2024-10-28 13:46:12Z",
      "referralUrl":"https://www.sample.co.uk/requests/item3.html"
    }    
  },
  "user":{
    "firstName":"John"
    "email":"john.smith@example.com"
  },
  "newsletterSubscription":true,
  "newsletterSubscribed":"2024-10-28 13:47:52Z",
  "consents":{
    "privacyPolicy":{
      "isConsentGranted":true,
      "lastUpdated":"2024-10-28 13:47:52Z"
    },
    "termsAndConditions":{
      "isConsentGranted":true,
      "lastUpdated":"2024-10-28 13:47:52Z"
    },
    "marketingCommunications":{
      "isConsentGranted":true,
      "lastUpdated":"2024-10-28 13:47:52Z"
    }
  }
}

What is Required?​

Looking at the data payload, not all attributes defined in the schemas are present. Only those required as indicated by e.g.

  "required":["firstName","email"],

in the schema, need to be submitted. Of course, it is possible to submit values for all attributes in the schema. However “required” represents both i) the minimum needed to have a valid object for business purposes and ii) the mimimum to ask for to reflect good data privacy practices.

info

One of the key principles of the GDPR is the concept of “data minimisation.” This means that organisations should only collect and process personal data that is necessary for a specific purpose

What is Asked for vs What is Required?​

In practice, the user supplied values are a subset of what is required in total. The remainder is either part of the form definition, calculated at runtime or inferred by the actions of the user.

To illustrate:

attributesourcecomment
metadata.localCampaign.localMarketCampaignIdform definition
metadata.localCampaign.campaignId.channelTypeIdform definition
metadata.localCampaign.campaignId.brandform definition
metadata.localCampaign.campaignId.classificationform definition
metadata.localCampaign.startDateform definition
metadata.localCampaign.endDateform definition---
metadata.localChannel.channelIdform definition---
metadata.localChannel.channelType.channelTypeIdform definition---
metadata.localChannel.nameform definition---
metadata.localChannel.descriptionform definition---
metadata.localChannel.referralUrlform definition---
metadata.localChannel.countryCodeform definition---
metadata.localChannel.languageCodeform definition---
metadata.formInstance.idform definition or runtime---
metadata.formInstance.createdByform definition---
metadata.formInstance.createdDateform definition---
metadata.formInstance.renderedDateruntimeform page load datetime
metadata.formInstance.referralUrlform definition or runtime---
user.firstNameuser suppliedtext entry
user.emailuser suppliedtext entry (email format validation)
newsletterSubscriptionInferredif the user fills out the newsletter sign-up form, we infer they want to subscribe (value equal to true). Later they may unsubscribe (setting the value to false)
newsletterSubscribedruntimeform submission datetime
consents.privacyPolicy.isConsentGranteduser suppliedcheckbox
consents.privacyPolicy.lastUpdatedruntimeform submission datetime
consents.termsAndConditions.isConsentGranteduser suppliedcheckbox
consents.termsAndConditions.lastUpdatedruntimeform submission datetime
consents.marketingCommunications.isConsentGranteduser suppliedcheckbox
consents.marketingCommunications.lastUpdatedruntimeform submission datetime

Presentation and User Experience​

The comments above indicating text boxes and checkboxes lead into the presentation and UX of the form.

The accessibility, usability and presentation of a webform is an important aspect of the overall form.

Please refer to TBD for more details on the aspects including

  • form entry validation

  • list of permissible values

  • form layouts

  • presentation on the webpage

  • accessibility requirements

Data Visibility​

There is also the fundamental aspect of which data is present and visible

  • in the form rendered in the client

    • visible fields
    • hidden fields

  • retained only on the server

Client Rendering - Visible​

These attributes fall into either

  • user editable fields (either blank/unselected or pre-filled/pre-selected)
  • user read-only fields

Business rules determine which are editable by the end user

Client Rendering - Hidden​

These attributes are passed to the client in the rendered form HTML, but are not actually displayed to the user in the browser. These are non-sensitive values which are required to process the form and its submission e.g. the campaign and channel IDs, but have no meaning/value for the end-user.

Different End-points​

Whilst it is common to submit the data submission payloads back to the endpoint providing the form, there are plenty examples where the data is submitted to a different endpoint. In this scenario, the subset of attributes to be sent to the client may need to be larger since the submission endpoint may lack the context of of the endpoint providing the form.

Client-side for Analytics and Auditing​

Initially, the focus tends to be on handling attributes directly related to the process of capturing form data submissions. However, it is also important to ensure that a webform is well enough defined, in-situ on the webpage, in order to meet business needs in terms of managing the campaigns, the websites etc. the sheer volume of content means it is not possible to manually audit webforms, and instead the auditing needs to be automated.

The automated auditing will be assessing forms from multiple perspectives

  • existence of the form
  • metadata of the form
  • form input fields and their labels
  • form accessibility
  • form usability
  • form security

Whilst that list is broad, specifically in terms of form definition, the focus is on

  • form metadata
  • form input fields and their labels

Like the scenario of data submission going to a different endpoint, the auditing tool will lack some of the context. To address this, some metadata is expected to be present in the form. As a minimum, the webform should contain the following metadata attributes

| attribute | source | comment | | metadata.localCampaign.localMarketCampaignId | form definition | --- | | metadata.localCampaign.campaignId.brand | form definition | --- | | metadata.localChannel.channelId | form definition | --- | | metadata.localChannel.countryCode | form definition | --- | | metadata.localChannel.languageCode | form definition | --- | | metadata.formInstance.id | form definition or runtime | --- | | metadata.formInstance.referralUrl | form definition or runtime | --- |

Plus it should be possible to clearly identify the form input fields by name, in English, following the standard naming conventions as far as possible i.e. do not replace “firstName” with “first_name” etc.

Server-side Only​

There are a further set of attributes which do not need to be delivered to the client, unless we consider the scenario where the payload is being submitted to a different endpoint to the one providing the form.

For example, “countryCode” and “languageCode” may be unnecessary to be sent to the client, as their values are obvious. Yet, as part of the “data transformation” stage, before storing/forwarding the response, these values should be added to ensure the final payload is a complete representation of the data submitted along with the metadata etc. providing the context.

Client​

However, beyond the basic form rendering/data submission, there is a further consideration