BaseLine Batch

BaseLine Batch can alert you to any security changes at any of the email addresses you send to. It is the third step in using CheckTLS to meet your email security needs:

1. Check your own email system.
   Use the More Options features on the two tests on our home page, Check How You Send Email and Check How You Get Email to thoroughly test your email.
2. Check email addresses you send to.
   Use CheckTLS "Batches" to run //email/testTo: ("TestReceiver") on a number of email addresses and only report on a few details like ConfidenceFactor, TLS Version, etc.
3. Know if anything changes.
   
   • For your own email system, monitor a Thru BatchTest targeting your own Domain.
   • For all other email addresses, monitor all the addresses you send to with BaseLine Batch.
   "Screws fall out all the time, the world is an imperfect place."

That last step is important because you are responsible for protecting mail you send (See "The Responsible Party" section of Internet Email Security Made Simple)
and
your emailer will send plain text if the recipient asks you to (prove this with //email/testMandatoryFrom:).

BaseLine Batch

With BaseLine Batch, you run your Batch of email addresses once and tell CheckTLS to save the results. Then when you run it again, you only see the differences between today and the saved results (the "BaseLine").

BaseLine Batch makes checking email addresses 1000 times faster. Instead of periodically looking at every address, only look at the ones that have changed. A tuned BaseLine Batch typically flags one out of a thousand addresses.

At any time you can update a BaseLine score, so if a target fixes or improves their email, you can set the new score as their BaseLine.

With BaseLine Batch, you:

1. list all the email addresses you send to (just the Domain part, the "Target")
2. test the addresses and split the results into "Good", "Bad", and "Untestable" Batches
3. run the Batches once to set the "BaseLine" scores
4. change the Batches to compare BaseLines
5. tune the Batches
6. run them regularly

These steps are described below.

Using BaseLine Batch

BaseLine Batch uses a lot of XML files. Microsoft Notepad will edit XML files, or download Microsoft's XML Notepad (required Edge to download) or Microsoft's more advanced XML Copy Editor.

CheckTLS can upload an Excel workbook that makes creating and maintaining the XML Batch source file easier. See //email/excelBatch.

We suggest you follow the steps below with raw XML files using the example Targets, but use Excel workbooks to work with your production tests.

1. list all the email addresses you send to

Use the New button in //email/testBatch to create a Batch of all the targets you want to BaseLine: <BatchTest TestType="receiverquick"> <Target>CheckTLS.com</Target> <Target>Invalid.CheckTLS.com</Target> <Target>RefuseTLS.CheckTLS.com</Target> <Target>TLSv1.CheckTLS.com</Target> <Delivery> <Format>csv</Format> </Delivery> </BatchTest>

In production, we recommend limiting a Batch to about 5000 Targets and using multiple Batches if you have more that.

2. test the addresses once and split the results

Run the Batch (Run Now button in //email/testBatch) and wait for it to finish (Show Queue button).

split the Targets into three new Batches:

• Download the csv results.
• Open the csv in Excel and sort the results by Score:

  -1

  Invalid Targets that are not valid email domains so could never be sent to. Fix them (typos) or remove them.

  0-49

  Bad Targets (did not do TLS)
  You should not be sending any confidential or protected information via email to these. Any email sent to them is going over the Internet in plain-text, which is likely illegal.
  Put them in a Bad Batch. When/if one becomes secure and you start sending to it, you can move it to a Good Batch.

  50

  Unreachable Targets that we could not test. These could be typos or intermittent failures (see Vagaries below).
  Remove any invalid Targets. Move Targets that are consistently not testable to an Untestable Batch. Good Targets that are only infrequently unreachable should stay in the Good Batch.

  51-87

  Bad Targets (did weak TLS)
  You should not be sending any confidential or protected information via email to these. Email sent to them is encrypted but the encryption is easily broken.
  Move them to a different Batch (Bad) and test them on a different schedule. When/if one becomes secure and you start sending to it, you can move it to a Good Batch.

  88-100+

  Good Targets (did good or very good TLS)
  These email systems use good (TLS v1.2) or very good (TLS v1.3) encryption.
  Test these frequently to be sure they stay secure and all email you send to them is safe and legal. When/if one becomes "Bad" you should stop sending any email to that domain and move it to your Bad Batch until it is fixed.

• If you have questions about a Target's score, use the interactive //email/testTo: test to look at the details.
• These Batches should look like this: <BatchTest TestType="setbaseline"> <Target>CheckTLS.com</Target> <Target>Invalid.CheckTLS.com</Target> <Target>Refuse.CheckTLS.com</Target> <Target>TLSv1.CheckTLS.com</Target> <Delivery> <Format>csv</Format> </Delivery> </BatchTest> If you are working through this page to learn BaseLine Batch, we suggest you just create one Batch with all the example Targets for now.
• Note the TestType is "setbaseline" and the Format is csv.

3. run the Batches once to set the "BaseLine"

Run the Batches and wait for them to finish.

4. change the Batches to compare BaseLines

Switch the Batches from setting the BaseLine to checking the BaseLine by changing the TestType to "baseline": <BatchTest TestType="baseline"> <Target>CheckTLS.com</Target> <Target>Invalid.CheckTLS.com</Target> <Target>Refuse.CheckTLS.com</Target> <Target>TLSv1.CheckTLS.com</Target> <Delivery> <Format>csv</Format> </Delivery> </BatchTest>

Running the Batches now will compare the current state (Score) of each Target with the saved BaseLine state. The result is just the targets that have changed: "Target","BaseLineScore","CurrentScore","Match" "WentBad.com","110","0","0" "WentGood.com","57","121","0" You can also use <Format>xml</Format>: <CheckTLS> <Results test="BatchTest_baseline" version="V03.17.09" format="xml" id="2" description="BaseLine compare" > <Result origin="www11-do.checktls.com"> <eMailAddress>WentBad.com</eMailAddress> <BaseLineScore>110</BaseLineScore> <CurrentScore>0</CurrentScore> <Match>0</Match> <eMailAddress>WentGood.com</eMailAddress> <BaseLineScore>57</BaseLineScore> <CurrentScore>121</CurrentScore> <Match>0</Match> </Result> </Results> </CheckTLS> Notice BaseLine test has a Match node so you do not have to parse and compare scores.

5. tune the Batches

Taking time to tune the BaseLine will save time in the future and improve your results.

If you are working through this page to learn BaseLine Batch, we suggest you use //email/editBaseLines to change the BaseLine score for the example Targets and run the BaseLine Batch to see the effect. For example if you change the BaseLine score for TLSv1.CheckTLS.com to 100 ("good") and CheckTLS.com to 0 ("bad"), running the BaseLine Batch will show that TLSv1.CheckTLS.com was "good" and is now "bad" and CheckTLS.com was "bad" and is now "good".

Once you have run a BaseLine, the following tuning can improve your results:

• Remove Targets that frequently don't match their BaseLine by editing the Batch in //email/testBatch.
• Allow several scores or score ranges to match in //email/editBaseLines.
• Keep Targets in their proper Batch based on their type or purpose (see Types of BaseLines below).
• Removed Targets will be automatically removed from the BaseLine when you next run it.
• Infrequently unreachable (Score=50) Targets should be removed from the stored BaseLine Scores but not removed from the Batch. That way they will still test despite being intermittently untestable. Use the RangeDelete option in //email/editBaseLines to remove all Targets with a score of 50.
• Changed Targets should be updated in //email/editBaseLines. This where the matching scores are stored.
• If you make a lot of changes to the Batch you should re-run a "setbaseline" Batch to save a new BaseLine. Re-running a "setbaseline" Batch will remove all the old saved BaseLine scores and replace them with current scores.

For really large Batches (thousands of Targets), see the next section "BaseLine Batch Auto-Tune".

6. run them regularly

When you want to compare the present settings with your saved BaseLine, run the Batch. As above, the result will be just the targets that have changed.

Periodically, if not every time, you should tune the BaseLine with new results. If a Target changes you should remove the old Match and add the new one, so future BaseLines will BaseLine against the Target's new Score.

If you ever want to completely reload all the BaseLines from this Batch, just reset the Format value and add the BaseLine attribute back and run the Batch. Just don't forget to put them back or you will replace the BaseLine every run instead of comparing to the BaseLine.

Moving to Production

When you have your BaseLine Batches tuned to only report problems, you can tell the Batches to only send a results email if something changed: either a Production ("Good") Target going bad, or a bad Target getting fixed. See Suppress in About Batch.

BaseLine Batch Auto-Tune

Auto-Tune automates the process of tuning large Batches. It removes mis-matches (failed BaseLine Targets) by adding new allowed matching scores for the mis-matches.

This is similar to re-running the Batch from scratch to reset all the BaseLines, but instead of replacing all the scores it adds new ones for any Targets that need them.

You should review the results of an automatic tune to make sure you don't inadvertently add a score that you do not want to accept. Targets effectively disappear from your exception report when you add their Score as a match.

To Auto-Tune a BaseLine Batch:

• You must have run at least one BaseLine (i.e. a Batch with TestType set to "baseline" -- see above).
• You must format the result as xml (i.e. <Format>xml</Format> -- see above).
• Use the AutoTune button in //email/editBaseLines to open the AutoTune screen.
• The Minimum Score depends on the purpose of this BaseLine (see Types of BaseLines below):

  Good Email

  88 and above (does good encryption)

  Bad Email

  51 to 87 (does weak encryption)

  Untestable Email

  50 (unreachable, therefore untested)

  Bad Email

  0 to 49 (does not do encryption)

• Check the XML Online box. This will use the copy of your results that we have stored on our servers.
  See below for how to manually fine tune the Auto-Tune.
• Click the RunTune button to perform the Auto-Tune.
• Your screen will show a summary of the new BaseLine scores that Auto-Tune added in the Test Results section under the buttons: added: weakdomain.com = 90 added: anotherweakdomain.com = 85; ... skipping duplicate: gooddomain.com = 90..100 added 274 records
• Next time you run the BaseLine, the mis-matches you had will be gone.

Manual BaseLine Batch Auto-Tune

To Manually Auto-Tune a BaseLine Batch:

• Download the last BaseLine Batch results (//email/testBatch:last).
  Not the original run, but an xml run, i.e. one that looks like this: <Result> <eMailAddress>weakdomain.com</eMailAddress> <BaseLineScore>60</BaseLineScore> <CurrentScore>90</CurrentScore> <Match>0</Match> </Result>
• Load the results file into Excel, sort by CurrentScore, and note any lines you don't want to update.
• Remove the records in an editor that understands XML (Excel will not let you save XML) or any text editor if you are careful to preserve the proper XML formatting.
• Use the AutoTune button in //email/editBaseLines to upload your edited results file and then to run it.
• Your screen will show a summary of the new BaseLine scores that Auto-Tune added in the Test Results section under the buttons: added: weakdomain.com = 90 skipping duplicate: gooddomain.com = 90..100 added 1 record
• Next time you run the BaseLine, the mis-matches you specified will be gone.

Types of BaseLines

We suggest you group email addresses ("Targets") that you want to test into three types:

• Good
• Bad
• Untestable

Remember that our Confidence Factor does not score the email address; rather it specifically scores the security of the email address.

Good Email

A BaseLine of Good Targets is a list of email addresses that you use that must be encrypted. You want to know if ever any one of them "breaks" and stops accepting encrypted emails. If one does break, you know to stop using it and contact them (maybe before they even realize they have a problem). If the Target is broken for a long time you may have to move it to one of the below BaseLines.

These should have Match Scores of 90 or above. Why accept a score that is measurably insecure? If you do find a Target below 90 that you want to accept, we allow you to add it.

Bad Email

A BaseLine of Bad Targets is a list of email addresses that you are not using because they are not encrypted. You want to know if ever any one of them gets "fixed" and starts accepting encrypted emails. Once they are OK you can let your organization know they can start using these email addresses. Then you should move fixed addresses to a Good BaseLine to make sure it stays secure.

These have Match Scores of 0 (zero) to 49. Note 50 is a special case below.

Untestable Email

A BaseLine of Untestable Targets is a list of email addresses that CheckTLS is unable to test. We don't know if they are secure or not, so we give them a "fifty fifty" chance. You want to know if ever any one of them gets "fixed" so CheckTLS can reach them and start testing them.

These have Match Scores of exactly 50.

Working Around Email Vagaries

Internet Email is designed to "never lose an email" and "get the mail through". It has enough redundancies to make sending any one email an adventure, as two emails sent at the same time can go two different ways. That makes it impossible to answer the question "Will every email ever sent to XYZ.com be properly encrypted?"

Which is why CheckTLS created the ConfidenceFactor. It takes everything into consideration to answer the question "How good is XYZ.com's email security?"

We call all the differences that an email system can exhibit "Vagaries".

Vagaries can result in a "false report" of a BaseLine change. The most common "false report" is when an email server gets busy and tells the sender to "try again later". This is expressly allowed by the formal email standard, and the sender is instructed to just try again in a few minutes. But our real-time test reports that as a failure to connect.

Other "false reports" come from one server in a redundant server pool having slightly different settings, or catching a site in the process of routine maintenance, etc. All these result in a slightly different Confidence Factor.

When doing BaseLine testing, we recommend TestType="receiverquick". "receiverquick" uses the Quick option in //email/testTo: to test only the one most likely MX host for a domain. This eliminiates most Vagaries.

Between any two BaseLine "receiverquick" runs we see less than 0.5% mis-matched Confidence Factors. So CheckTLS BaseLine testing makes checking your trading partners' emails 200 times easier.

A few simple tune-ups can reduce false reports to one in a thousand or less:

• Use //email/editBaseLines to set the score or scores that are acceptable.
  • Manually set the score
  • Add two or more scores
  • Add a range of scores
• Move any Targets with frequent "false positives" to a separate Batch, or remove them altogether. BaseLine doesn't work well for Targets that change all the time.

Advanced Features

BaseLine testing has some additional features:

Multiple Scores and Score Minimums and Maximums

//email/editBaseLines lets you list two or more scores that will match a Target. It also lets you set a scoring range that will match, so for example you can use the range 88 to 121 to mean "Yes this domain uses TLS 1.2 or better".

Multiple BaseLines for a Single Targets

The standard //email/testTo: ("TestReceiver") test has many options. BaseLine Batch lets you check the same Targets with different options. For example, you can BaseLine the Targets's security with TLS version 1.2 and separately with TLS version 1.3: <Target SSLVersion="TLSv1_2">V1.2@XYZ.com</Target> <Target SSLVersion="TLSv1_3">V1.3@XYZ.com</Target> This will show you when XYZ.com changes from TLS v1.2 to TLS v1.3.

BaseLine Batch keeps the full Target email address so you can BaseLine the same Targets with different settings.

BaseLine and Details in the Same Batch

With a simple hack you can use a BaseLine Batch to show details like SSLVersion and Cert expiration for the same Targets that are in the BaseLine. Keep two <Delivery> nodes at the bottom of the Batch, one that makes it a BaseLine and one that makes it show details. Comment out one or the other to control what the Batch does. For example: <BatchTest TestType="receiverquick"> <Target>checktls.com</Target> <Target>Dori.com</Target> <Target>Nori.com</Target> <Target>Ori.com</Target> <Delivery> <Format>xml</Format> </Delivery> <!-- <Delivery> <To>file</To> <Format>xml-certdetail</Format> <OnlyNode>eMailAddress</OnlyNode> <OnlyNode>ConfidenceQFactor</OnlyNode> <OnlyNode>//MX[1]/@exchange</OnlyNode> <OnlyNode>//MX[1]/SSL/SSLVersion</OnlyNode> <OnlyNode>//MX[1]/SSL/Cipher</OnlyNode> <OnlyNode>//MX[1]/SSL/Certs/Cert[@number=1]/NotValidAfter</OnlyNode> </Delivery> --> </BatchTest>

WebService (API)

You can maintain the stored BaseLines for a Batch by using BaseLineEdit as a WebService.

ADD A TARGET:

https://www.CheckTLS.com/BaseLineEdit ?COMPANYCODE=me@mydomain.com &COMPANYPASS=IllNeverTell &BATCHID=52 &TARGET=three@checktls.com &SCORE=99 &ACTION=wsAddTarget

Success returns: <Results> <Result>Success: three@checktls.com(#52) = 99</Result> </Results>

Any errors returns: <Errors> <Error>Invalid COMPANYCODE or COMPANYPASS: See FAQ</Error> <Error>Invalid BATCHID: 1a</Error> <Error>Invalid TARGET: (blank)</Error> <Error>Invalid SCORE: alpha</Error> </Errors>

DELETE A TARGET:

https://www.CheckTLS.com/BaseLineEdit ?COMPANYCODE=me@mydomain.com &COMPANYPASS=IllNeverTell &BATCHID=52 &TARGET=three@checktls.com &ACTION=wsDeleteTarget

Success returns: <Results> <Result>Success: 1 record deleted</Result> </Results>

DELETE ALL TARGETS:

https://www.CheckTLS.com/BaseLineEdit ?COMPANYCODE=me@mydomain.com &COMPANYPASS=IllNeverTell &BATCHID=52 &ACTION=wsDeleteAllTargets

DELETE BY SCORE:

Delete all records with a ScoreMin between 0 and 10 and a ScoreMax between 20 and 30: https://www.CheckTLS.com/BaseLineEdit ?COMPANYCODE=me@mydomain.com &COMPANYPASS=IllNeverTell &BATCHID=52 &SCOREMIN=0..10 &SCOREMAX=20..30 &ACTION=wsDeleteScores

To delete all records with a ScoreMin below 50 use: &SCOREMIN=0..50 &SCOREMAX=0..999

Success returns: <Results> <Result>Success: 437 records deleted</Result> </Results>