Conditions and Terms of Use

Microsoft Confidential
This training package is proprietary and confidential, and is intended only for uses described in the training materials. Content and software is provided to you under
a Non-Disclosure Agreement and cannot be distributed. Copying or disclosing all or any portion of the content and/or software included in such packages is strictly
prohibited.
The contents of this package are for informational and training purposes only and are provided "as is" without warranty of any kind, whether express or implied,
including but not limited to the implied warranties of merchantability, fitness for a particular purpose, and non-infringement.
Training package content, including URLs and other Internet website references, is subject to change without notice. Because Microsoft must respond to changing
market conditions, the content should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any
information presented after the date of publication. Unless otherwise noted, the companies, organizations, products, domain names, e-mail addresses, logos,
people, places, and events depicted herein are fictitious, and no association with any real company, organization, product, domain name, e-mail address, logo,
person, place, or event is intended or should be inferred.

Copyright and Trademarks

© 2014 Microsoft Corporation. All rights reserved.
Microsoft may have patents, patent applications, trademarks, copyrights, or other intellectual property rights covering subject matter in this document. Except as
expressly provided in written license agreement from Microsoft, the furnishing of this document does not give you any license to these patents, trademarks,
copyrights, or other intellectual property.
Complying with all applicable copyright laws is the responsibility of the user. Without limiting the rights under copyright, no part of this document may be
reproduced, stored in or introduced into a retrieval system, or transmitted in any form or by any means (electronic, mechanical, photocopying, recording, or
otherwise), or for any purpose, without the express written permission of Microsoft Corporation.
For more information, see Use of Microsoft Copyrighted Content at
http://www.microsoft.com/about/legal/permissions/
Microsoft®, Internet Explorer®, Outlook®, OneDrive®, Windows Vista®, Zune®, Xbox 360®, DirectX®, Windows Server® and Windows® are either registered
trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. Other Microsoft products mentioned herein may be either
registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries. All other trademarks are property of their respective
owners.
http://aka.ms/easGWT
 Exchange Remote Connectivity Analyzer
http://aka.ms/exrca – Did test succeed?

UPN match
RCA results primary
identify SMTP?
failure? Mailbox
7-8 Policy test

1-5
7-10 Maybe the issue is 6
related to connectivity?
Finding the right server
Exchange 2013/2016/2019 begin at the Mailbox server where the mailbox is located

get-mailbox alias | ft servername,alias

Finding the right server
Finding the right server
Exchange 2013, 2016 and 2019 have the Managed Availability service that monitors ActiveSync health

• Open the browser on mobile device

• Browse to https://mail.contoso.com/Microsoft-Server-ActiveSync/HealthCheck.htm

• Returns the CAS FQDN and HTTP status code indicating health

• This shows the entry point for the client connection and can be used to retrieve HttpProxy and IIS
logs
Finding the right server
PowerShell log search

• Be sure to flush the logs to ensure latest request is written from memory and can be searched

netsh http flush log buffer

• Make sure to use the logs from the correct directory, W3SVC1 for front end, W3SVC2 for backend

Accurate

Select-String –Path c:\inetpub\logs\LogFiles\W3SVC1\u_exyymmdd.log –Pattern

alias | Select-String –Pattern ActiveSync | Select-Object –Last 10

Fast

Get-Content “C:\inetpub\logs\LogFiles\W3SVC2\u_exyymmdd_x.log” –Tail 5000 |

select-string alias
Exchange ActiveSync mailbox logs
• Per mailbox, not device

• Web.config controls more granular logging options

• Records headers, SOAP requests and response pairs

• Server perspective of client communication

• Logging disabled automatically after 72 hours on prem, or 24 hours in Exchange Online

• Overwrites using FIFO when log size reaches threshold

Exchange ActiveSync mailbox logs
To enable logging for a mailbox:

Set-CASMailbox alias –ActiveSyncDebugLogging:$True

To retrieve logs for all devices associated with a mailbox

Get-MobileDeviceStatistics –Mailbox alias –GetMailboxLog:$True –

NotificationEmailAddresses admin@contoso.com

To retrieve logs for a specific device

Get-MobileDeviceStatistics <DN of device> -GetMailboxLog:$True | Select –

ExpandProperty MailboxLogReport | Out-File C:\Temp\Device.log
<?xml version="1.0" encoding="utf-8"?>
<Autodiscover
xmlns="http://schemas.microsoft.com/exchange/autodiscover/mobilesync/requestschema/2006">
<Request>
<EMailAddress>jim@tailspintoys.com</EMailAddress>
<AcceptableResponseSchema>http://schemas.microsoft.com/exchange/autodiscover
/mobilesync/responseschema/2006</AcceptableResponseSchema>
</Request>
</Autodiscover>"
<?xml version="1.0" encoding="utf-8"?>
<Autodiscover xmlns="http://schemas.microsoft.com/exchange/autodiscover/responseschema/2006">
<Response xmlns="http://schemas.microsoft.com/exchange/autodiscover/mobilesync/responseschema/2006">
<Culture>en:us</Culture>
<User>
<DisplayName>Jim Martin</DisplayName>
<EMailAddress>jim@tailspintoys.com</EMailAddress>
</User> <Action>
<Settings>
<Server>
<Type>MobileSync</Type>
<Url>https://eas.tailspintoys.com/Microsoft-Server-ActiveSync</Url>
<Name>https://eas. tailspintoys.com/Microsoft-Server-ActiveSync</Name>
</Server>
</Settings>
</Action>
</Response>
</Autodiscover>"
• OPTIONS
• FolderSync or Settings
• Provision
• Provision
• FolderSync
• Sync
• Supported EAS protocol versions
• Supported EAS commands

Response:

Supported Versions: 2.0,2.1,2.5,12.0,12.1,14.0,14.1

Highest Supported Version: 14.1
Supported Commands: Sync,SendMail,SmartForward,SmartReply,GetAttachment,GetHierarchy,CreateCollection,
DeleteCollection,MoveCollection,FolderSync,FolderCreate,FolderDelete,FolderUpdate,MoveItems,GetItemEstimate,
MeetingResponse,Search,Settings,Ping,ItemOperations,Provision,ResolveRecipients,ValidateCert
RequestBody : ResponseBody :
<?xml version="1.0" encoding="utf-8" ?> <?xml version="1.0" encoding="utf-8" ?>
<FolderSync xmlns="FolderHierarchy:"> <FolderSync xmlns="FolderHierarchy:">
<SyncKey>0</SyncKey> <Status>142</Status>
</FolderSync> </FolderSync>
RequestBody : ResponseBody :
<?xml version="1.0" encoding="utf-8" ?> <?xml version="1.0" encoding="utf-8" ?>
<Provision xmlns="Provision:"> <Provision xmlns="Provision:">
<DeviceInformation xmlns="Settings:"> <DeviceInformation xmlns="Settings:">
<Set> <Status>1</Status>
<Model>GT-I9295</Model> </DeviceInformation>
<IMEI>358194052013256</IMEI> <Status>1</Status>
<FriendlyName>GT-I9295</FriendlyName> <Policies>
<OS>Android 4.4.2</OS> <Policy>
<UserAgent>Android/4.4.2</UserAgent> <PolicyType>MS-EAS-Provisioning-WBXML</PolicyType>
</Set> <Status>1</Status>
</DeviceInformation> <PolicyKey>2000257276</PolicyKey>
<Policies> <Data bytes="53"/>
<Policy> </Policy>
<PolicyType>MS-EAS-Provisioning-WBXML</PolicyType> </Policies>
</Policy> </Provision>
</Policies>
</Provision>
Requestbody : Responsebody :
<?Xml version="1.0" encoding="utf-8" ?> <?Xml version="1.0" encoding="utf-8" ?>
<provision xmlns="provision:"> <provision xmlns="provision:">
<Policies> <Status>1</status>
<Policy> <Policies>
<Policytype>ms-eas-provisioning-wbxml</policytype> <Policy>
<Policykey>2000257276</policykey> <Policytype>ms-eas-provisioning-wbxml</policytype>
<Status>2</status> <Status>1</status>
</Policy> <Policykey>72162277</policykey>
</Policies> </Policy>
</Provision> </Policies>
</Provision>
Requestbody : Responsebody :
<?Xml version="1.0" encoding="utf-8" ?> <?Xml version="1.0" encoding="utf-8" ?>
<foldersync xmlns="folderhierarchy:"> <foldersync xmlns="folderhierarchy:">
<Synckey>0</synckey> <Status>1</status>
</Foldersync> <Synckey>1</synckey>
<Changes>
<Count>19</count>
<Add>
<Serverid>1</serverid>
<Parentid>0</parentid>
<Displayname>calendar</displayname>
<Type>8</type>
</Add>
<Add>
<Serverid>5</serverid>
…
Requestbody : Responsebody :
<?Xml version="1.0" encoding="utf-8" ?> <?Xml version="1.0" encoding="utf-8" ?>
<sync xmlns="airsync:"> <sync xmlns="airsync:">
<Collections> <Collections>
<Collection> <Collection>
<Synckey>0</synckey> <Synckey>274695188</synckey>
<Collectionid>5</collectionid> <Collectionid>5</collectionid>
</Collection> <Status>1</status>
</Collections> </Collection>
</Sync> </Collections>
</Sync>
• C re a t e s E x c h a n g e A c t i v e S y n c D e v i c e s l e a f o b j e c t c o n t a i n e r u n d e r u s e r o b j e c t i n
A c t i v e D i re c t o r y
• C re a t e s a m s E x c h A c t i v e S y n c D e v i c e o b j e c t f o r e a c h a c t i v e s y n c c l i e n t i n a f o l d e r
c re a t e d w i t h i n m a i l b ox :
ro o t C o n t a i n e r \ E x c h a n g e S y n c D a t a \ < D e v i c e _ I d e n t i t y >
Process flow

Is the device authenticated?

Is ActiveSync enabled for the user?
Is the mailbox policy enforced by the device?
Is the device blocked for the mailbox?
Is the device exceeding an autoblock threshold?
Is the device allowed for the mailbox?
Is the device blocked by a device access rule?
Is the device quarantined by a device access rule?
Is the device allowed by a device access rule?
Apply the default access state [Quarantine allows device connection in limited capacity]
quarantine new unknown
devices
Info sent to users when mobile
device isn't syncd because
device is quarantined.
Notification Address is also
sent details of device
quarantined
Set-ActiveSyncOrganizationSettings
-DefaultAccessLevel Quarantine
-UserMailInsert "Your device is currently undergoing
corporate validation, please be patient while we confirm
that this new device meets our standards.”
-AdminMailRecipients
mobileadmin@yourorg.onmicrosoft.com
35
Mailbox

Get-CASMailbox
• ActiveSyncEnabled
• ActiveSyncBlockedDeviceIDs
• ActiveSyncAllowedDeviceIDs

Get-MobileDeviceStatistics
• DevicePolicyApplied
• DevicePolicyAppliedStatus
Global

Get-ActiveSyncDeviceAccessRule
• QueryString
• Characteristic
• AccessLevel

Get-ActiveSyncOrganizationSettings
• DefaultAccessLevel
Example

Explanation: Allowed 300 commands per 5 minutes. If hit, block for 2 minutes and send an email to the user’s mailbox with
the custom explanation.
• Requests are not instantaneously logged
• The date-time is in GMT
• The timestamp of the logs and when logs roll can vary
• Logs can get very large (over 1GB per day per server is common)
• Exchange 2013 and 2016 have IIS logs for two web sites (Default Web Site and Exchange Backend
Site)
• Know you status codes http://support2.microsoft.com/default.aspx?scid=kb;EN-US;943891
Status code Description Caveats
200 OK Does not indicate EAS command success

400 Bad request Access token bloat

401 Unauthorized Wrong password, misconfigurd MDM

403 Forbidden SSL config error, CBA

404 Not found
441 Missing CSC cache entry Default timeout 20 seconds
449 Need provisioning Policy refresh or policy change
451 Redirect
500 Internal server error
503 Service unavailable Application level outage, ABQ
• Most field data is typical web server information
• Cs-uri-query element is verbose EAS logging information
• Sc-win32-status can give more information on HTTP status error
Example: net helpmsg 1909
The referenced account is currently locked out and may not be logged on to.
cs-uri-query translation
• & and _ used to separate data
• HTTP escaping characters used
Example: %3a represents “:” or PrxFrom:%3a%3a1 would be Proxy from ::1 which is the loopback address
• Fc = folder count
• Fid – FolderID
• Sk = SyncKey
• Correlates to IIS log entry by RequestId (Default Web Site)
• TotalRequestTime may be high for hanging Ping or Sync commands (heartbeat interval)
• 42 fields
• Last field is GenericError which may give more information on connection issues
• Versatile tool used to analyze many different types of logs
• Good for troubleshooting individual or macro ActiveSync issues
• ActiveSync troubleshooting primarily uses IIS logs
• Build in library includes queries
• Provides UI for powerful LogParser engine
Specific strategies for EAS troubleshooting
• GWT shows specific queries
• Track individuals by username or device
• Show EAS error codes and messages
• SyncKey 0 issues
 1. import multiple logs merging them
2. flatten Activesync Mailbox Logs into table view
3. search all entries
4. displays original Log Entry for detailed view

1.
2.
3.
4.
Common Status codes for all commands [status code value > 100]
https://msdn.microsoft.com/en-us/library/ee218647(v=EXCHG.80).aspx
At the bottom portion of the page
Commands specific status codes
https://msdn.microsoft.com/en-us/library/ee218647(v=EXCHG.80).aspx
Find the status element section link
How to translate CollectionIds to Mailbox folder names

• If the Activesync mailbox logs capture an initial FolderSync response, the ServerId to folder
names will be represented, where ServerId = CollectionId in sync messages.
• There is a method using MFCMAPI for on prem mailboxes
• There is an EWS script that translates both on prem and Exchange Online
• Using the Get-ExchangeDiagnosticInformation command on prem with the correct parameters
• Alphabetically can guestimate the mappings especially when no new folders have been added
to the default starting folders. For example, Calendar is typically first alphabetically in the folder
list, so its CollectionId is frequently = 1.
http://exrca.com
• With EASInspector addin, full
verbose SOAP
• This means subject, attendees
searchable
• Good for troubleshooting
individual device issues, including
performance
• Keep in mind, Ping and Empty
Sync requests will have expectedly
long server think times
https://technet.microsoft.com/en-
us/library/dd638102(v=exchg.160).aspx

https://technet.microsoft.com/en-
us/library/jj552406(v=exchg.160).aspx
 Calendar Diagnostic Logs
1. Available when Calendar version store is enabled
Get-mailbox alias | fl *calendar*
Mailbox
Meeting requests Calendar Version Store
sent or received

Inbox When meeting requests

are edited
Sent Items
Meeting requests or Deleted Items
calendar items moved
to deleted items Calendar
When calendar items
Dumpster 2.0 are modified
Deleted Items
folder emptied Recoverable Items

Deletions

Meeting items “purged” Purges

Versions
Calendaring issues

Data collection Methodology

• Detailed account of events leading up to symptom
• Calendar Diagnostic Logs
• Exchange Activesync Mailbox Logs
• IIS Logs
• Fiddler trace
• SARA results
• device specific logging*
Calendaring issues

What properties do Activesync calendar item use?

Global Object ID (GOID)
• Generated when the organizer sends the meeting request
• Used to match meeting updates and responses for a meeting
• Same across all copies of the calendar item
• Represented by the UID element in an EAS mailbox log
ServerID – unique identifier assigned to each object synchronized by Exchange
InstanceID – GMT timestamp of the original start time for this instance of the recurring meeting
Calendaring issues
Typical IIS log entries look like this, but we can’t get them easily from the cloud…

2015-01-20 13:36:43 fe80::74da:4feb:67d2:d395%14 POST /Microsoft-Server-

ActiveSync/Proxy/default.eas
User=jmartin@tailspintoys.com&DeviceId=ApplF7PLNQX0PF48&DeviceType=iPad&Cmd=Sync&Log=P
rxFrom:fe80%3a%3a74da%3a4feb%3a67d2%3ad395%2514_V141_HH:eas.tailspintoys.com
_SmtpAdrs:jmartin%40tailspintoys.com_Fc1_Fid:1_Ty:Ca_Filt5_St:S_Sk:79675391_Sks599688612_Sst1_SsC
mt2_TotSvC1_ColdSvC1_TotLdC1_MR0_GetChgsIter1_GetChgsTime3_Srv:0a0c1d0s0e0r0A0sd_Pfs1_BR1_
BPR0_Fet234_Pk3098698031_DevOS:iOS+8.1.2+12B440_S1_As:AllowedG_Mbx:CLT-EX13-
MBX1.tailspintoys.com_Throttle0_SBkOffD:BBkOff%3aL%2f-469%2c+ABBkOff%3aL%2f-
600%2c+EffBkOff%3aL%2f-
469_SyncHC311625842_TmRcv13:36:43.910384_TmSt13:36:43.910384_TmDASt13:36:43.9259636_TmPolS
t13:36:43.9259636_TmExSt13:36:43.9259636_TmExFin13:36:44.1290921_TmFin13:36:44.1447103_TmCmpl
<…SNIP…>
460d8d5dcafc%5d%3d90%3bDbl%3aST.T%5bCLT-EX13-MBX1.fcf8d606-5e15-4120-bed6-
460d8d5dcafc%5d%3d67%3bDbl%3aMBLB.T%5bCLT-EX13-MBX1.fcf8d606-5e15-4120-bed6-
ee-9110-7c836abbba13%2cIsServiceAccount%3aFalse%2cLiveTime%3a00%3a00%3a03.9062595_ 444
TOYS\jmartin fe80::74da:4feb:67d2:d395%14 Apple-iPad2C5/1202.440 - 200 0 0 14490 3675 249
Calendaring issues
IIS logs
Informational only – not enough details

Enable EAS mailbox logging

Logging automatically turns off after x number of hours
Scripts available to overcome this limitation

Scheduling collection of ActiveSync Mailbox logs -

https://gallery.technet.microsoft.com/Scheduled-collection-of-c97b3460
Exchange ActiveSync log parser for calendar items -
https://gallery.technet.microsoft.com/Exchange-ActiveSync-log-02b62a03

Wait for issue to reoccur

Global Object ID
Field name Type Size Sample Description
Byte Array ID BYTE array 16 04 00 00 00 82 00 E0 This byte array identifies the BLOB as a Global Object ID
00 74 C5 B7 10 1A 82
E0 08
Year WORD 2 00 00 The original year of the instance represented by the
exception.
M BYTE 1 00 The original month
D BYTE 1 00 The original day
Creation Time PtypTime 8 A9 06 1B E8 BB C3
D0 01
X BYTE array 8 00 00 00 00 00 00 00 Reserved
00
Size LONG 4 10 00 00 00 The length of the Data field
Data BYTE array 16 FA 82 B1 70 7C 3C CA The data that uniquely identifies this meeting object
4F 9B 0C 9A 1A AB
C9 F6 EF
040000008200E00074C5B7101A82E00800000000A9061BE8BBC3D001000000000000000010000000FA82B1707C3CCA4F9B0C9A1AABC9F6EF
 Sample
<ServerId>1:6</ServerId>
<ApplicationData>
<TimeZone>LAEAACgAVQ///w==</TimeZone>
<StartTime>20150105T170000Z</StartTime>
<Subject xmlns="Calendar:" bytes="25"/>
<UID>040000008200E000748B5DF416EA9</UID>
<EndTime>20150105T180000Z</EndTime>
<Recurrence>
<Type>1</Type>
<Interval>1</Interval>
<Until>20151116T170000Z</Until>
<DayOfWeek>2</DayOfWeek>
<FirstDayOfWeek>0</FirstDayOfWeek>
</Recurrence>
<Body=4 bytes/>
<Sensitivity xmlns="Calendar:">0</Sensitivity>
<BusyStatus xmlns="Calendar:">2</BusyStatus>
<AllDayEvent xmlns="Calendar:">0</AllDayEvent>
<Reminder xmlns="Calendar:"></Reminder>
Mail issues – Direct Push

T=O: If any items

change in the next 15
min. let me know.
Otherwise, return OK.
Enterprise Network
T=15: No response
…network must have dropped

T=15: If l receive
mail in the next 8
min. let me know.
Otherwise, return OK

T=23: HTTP 200 OK CAS

Cellular …network must be good try longer Servers
Network TCP TCP
T=23: If new items in the
443 443
next 12 min. let me know. AD
Otherwise, return OK.

T=26: New item in Inbox

T=26: Sync Inbox

Mailbox
T=23: If new items in the
next 12 min. let me know.
Servers
Otherwise, return OK.
Sync – Sample Request

<?xml version="1.0" encoding="utf-8" ?>

<Sync xmlns="AirSync:">
Performed <Collections>
Operation <Collection>
Current Sync Level <SyncKey>662762728</SyncKey>
(Watermark) on <CollectionId>21</CollectionId>
the device <GetChanges/>
<WindowSize>25</WindowSize>
Folder (e.g. Inbox <Options>
or Calendar) <FilterType>5</FilterType>

<MIMETruncation>1</MIMETruncation>

Microsoft Confidential 76
Sync – Sample Response

<?xml version="1.0" encoding="utf-8" ?>

<Sync xmlns="AirSync:">
<Collections>
<Collection>
Current sync level <SyncKey>1387514663</SyncKey>
(Watermark) on the
<CollectionId>21</CollectionId>
server – Replaces the
value on the device <Status>1</Status>
<Commands>
Operation <Add>
successful?
<ServerId>21:301</ServerId>

Microsoft Confidential 77
SyncKey = 10000002
<ApplicationData />

SyncKey = 10000003
<ApplicationData />

Status = 3 (Invalid SyncKey)

https://docs.microsoft.com/en-us/Exchange/architecture/client-access/client-message-size-limits?view=exchserver-2019
https://technet.microsoft.com/en-
us/library/ff459250(v=exchg.160).aspx
https://autodiscover.thejimmartin.com/autodiscover/autodiscover.json?email=bob@thejim
martin.com&Protocol=ActiveSync
Tokens
Access token Refresh token
• Specific resource • UPN specific
• Short lived (~1 hour) • Long lived (between 14-90 days)
• Stored in registry • Stored in Credential Store
HKCU\Software\Microsoft\Office\16.0\Common\
Identity\Identities\<GUID>_ADAL
Modern auth (Federated Identities)