AX AOS TROUBLESHOOTING TIPS

AOS Configuration Best Practices Troubleshooting AOS issues Finding Sessions and SPIDS AOS Crash Maintenance Tools for Troubleshooting AOS issues

Application Object Server Configuration Database Tuning Tab Tracing Tab Performance Tab (Dynamics AX 2009)

Dynamics AX Server Configuration Utility

Configuration stored in Registry:
HKLM\SYSTEM\CurrentControlSet\Dynamics Server\4.0\instance\config name

Multiple AOS instances and configurations

Each AOS instance listed Each AOS has configurations

Configuration command to run at startup Allowing clients to print from AOS printers Enabling breakpoints Fatal error reporting Load Balancing

Number of client sessions What is a load balancer How connections work when using load balancing Factors when deciding if you need a load balancer

Provides fail over ability Balances number of clients between AOS Control the number of clients per AOS

With no options marked in the load balancing section each client will connect to the AOS in their client configuration

As a client connects to an AOS that AOS queries the database to determine which AOS the client will connect to

The load balancer makes the connection to the database and determines which AOS the client will connect to This AOS will not accept any client connections nor do any processing This AOS does not count against your AOS licenses

In the Dynamics AX 4.0 Configuration utility you will want to specify the machine name and port number for the AOS load balancer instance Add AOS member instances after the load balancer

Max open cursors Max buffer size Allow index hints in queries

RPC round trips to server X++ method calls Function calls Start and Stop Tracing

SQL statements Row Fetch Connect and disconnect Transactions: TTSBegin,TTSCommit,TTSAbort Bind variables Row fetch summary

Performance tab 64 bit Batching Time Zones

Separate AOS Separation of environments Monitoring performance Code deployment Applying latest Kernel hotfixes

Consider separating out the following processes / components based on load

Additional Dynamics AX components Third Parties Batch

Servers for production environment should be on dedicated boxes

Server resources dedicated to production performance Reduce risk of development code utilizing majority of Servers resources Application of hotfixes Additional components that need .Net BC

Performance Monitor
Memory CPU

Memory usage
Different times of day When specific processes run

Code promotion schedule Promote code during down time Determine if you will use xpos or layers to move code from development/test to production Test code in environment that matches production before deploying

Improved Stability Apply on all machines with a Dynamics AX components

Dynamics AX client Enterprise Portal SSRS machine Any machine with .Net Business Connector

Application event log RPC errors Connection issues AOS Crashing

Windows default event logging tool Different types of events

AOS starting User errors Cache flushing Crashes

Lost connection to AOS Cannot or lost connection to SQL Lost connection to the AOS

Object Server 01: RPC error: Client provided an invalid session ID 75

This error message indicates that the AOS has crashed. After the AOS crashed a client that was open when the crash occurred tried to connect to the AOS. Since the AOS was restarted it does not recognize the existing client connection. To resolve this issue the client has to be shut down and restarted.

Object Server 01: No ping from 77. Terminating the session.

This error message indicates that the end user ended the client session before the AOS could return the requested results to the client. To prevent this error message end users shouldnt end task (ctrl alt delete) their Dynamics AX session. Instead an end user should contact their AX administrator and have them end their session from the online users form.

Object Server 01: SPID 57 for Session Id 77, is still present in the database. Please delete the SPID from the database.
This indicates that the end user shut down their Dynamics AX client before SQL finished processing the request. You can delete the SPID or let SQL finish processing If you dont clean up the SPID it could cause issues such as blocking transactions, and open transactions which may lead to transaction log backup failures.

Object Server 03: Fatal SQL condition during login. Error message: "[Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for user 'NORTHAMERICA\MBSAXAPTA04$'.
This typically indicates the AOS service account does not have permissions to the database.

*Typically followed by error on next slide

Object Server 03: SQL diagnostics: [Microsoft][ODBC SQL Server Driver][SQL Server]Login failed for user 'NORTHAMERICA\MBSAXAPTA04$'.. Connect information was: Userid = [], Database = [DAX4_SP1_Prod], Server = [server1], DSN = [], Other = [] This is caused by insufficient SQL permissions. To resolve this issue please use Knowledge base article 911380 method 2.

Object Server 03: Internal Error occurred executing stored procedure when creating session for the AOS.
This indicates the AOS Service account does not have adequate permissions to the stored procedures in the database. To resolve this issue please use Knowledge base article 911380 method 2..

Error in file: C:\Program Files \DynamicsAX\ Application\appl\prod \axsys.aod while reading in record 215B1D0 Windows Error: = Error Code: 64 = The specified network name is no longer available. Try again?
This error indicates the AOS has lost connection to the Application files. To resolve this error message you need to restart the AOS.

Find a user using a session ID in SQL Find a SPID using a session ID in SQL Finding User and computer from the AOT

If the following event was logged on 6/1/2009 you could run the query below. Object Server 01: No ping from 77. Terminating the session.
select * from sysuserlog where sessionid = 77 and createddate > '2009/06/01' order by createddate desc

1.

Check the status of the Microsoft Dynamics AX sessions, verify that these sessions have active SPIDs in the Microsoft Dynamics AX database where the client session is in the shut down process. To do this, run the following SQL statement:
Select * from SYSCLIENTSESSIONS where status = 2 or status = 3 In the results of the statement, check the matching session ID from the warning message that appears in the Application log on the AOS server

2.

Check the status of the orphaned SQL SPID. To do this, run the following SQL statement:
Select * from master..sysprocesses In the results of the statement, locate the SPID that is indicated in the warning message, and then verify the following:
If the SPID is blocked or if the SPID is blocking other SPIDs, and if the waittime column shows a high number, you can decide to delete the SPID If the open_trans column has a value of 1 for a SPID line, there is a pending transaction for the SPID line. Additionally, if the waittime column shows a high number for this SPID line, you should delete the SPID

Users should NOT end task (CTRLALT-DEL) their Dynamics AX session. Instead your users should:
Contact their AX administrator and have them end their session from the (Administration>Online) users form <Or> Users should CTRL-BREAK the process.
*This allows the AOS to gracefully close network connections and recover resources

RPC 1726 communication issue between client and server. Typically it doesnt mean you have a network error, but rather that the AOS was overloaded RPC 5 Access denied RPC 1702 -The binding handle is invalid

Options for collecting dump files Option in Dynamics AX Server Configuration How to get information to Microsoft

Automatically send reports to Microsoft How to use ADPlus to troubleshoot "hangs" and "crashes http://support.microsoft.com/kb/286350/en-us Debugging Tools for Windows Overview (Download) http://www.microsoft.com/whdc/devtools/debugging/default .mspx Debug Diagnostic Tool http://www.microsoft.com/downloads/details.aspx?FamilyI D=28bd5941-c458-46f1-b24df60151d875a3&displaylang=en

Mark the For all AOS instances in this computer, automatically send reports about fatal errors to Microsoft option on the Application Object Server tab in the Server Configuration Utility.

When AOS crashes you receive an event id of 1001 in application event log Here is an example of the error:

Contact Microsoft Support with contents of the application event log event id 1001 Support Technician passes bucket id on to Escalation Engineer so they can obtain file Technical Support may contact you for more information. Ie. Application files (*.AOD)

Performance Monitor Application event log Logging SQL Server tools Debug Diagnostic Tool

PAL tool Memory

Typically greatest cause of issues
Monitor VIRTUAL BYTES in 32 bit environment Monitor WORKING SET in 64 bit environment

CPU
Prior to Dynamics AX mostly an issue for batch processing In 2009 with 64 bit CPU may see more CPU usage than in the past

Monitor it
Have a schedule for monitoring

Sizing of it
Make sure events dont get overwritten

Schedule to clear it out

SQL Server Tracing in Dynamics AX

Easy way to get sql and X++

Client/Server Tracing
More advanced requires some setup and use of another tool to review information

Trace Parser
Used to review Client/Server Tracing

SQL Profiler
See log of what sql is doing

SQL Management Studio

Running queries Changing settings Maintenance

SQL DMVs

Debug Diagnostic Tool

Have it installed on AOS machines Have Hang collection setup

Debugging tools for Windows

If you want to review memory dump files

 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.