
What
One of the best ways to track down problems in the API runtime environment is to log messages. You can attach and configure a MessageLogging policy on your API to log custom messages to syslog.
Samples
Syslog
<MessageLogging name="LogToSyslog"> <Syslog> <Message>[3f509b58 tag="{organization.name}.{apiproxy.name}.{environment.name}"] Weather request for WOEID {request.queryparam.w}.</Message> <Host>logs-01.loggly.com</Host> <Port>514</Port> <Protocol>TCP</Protocol> <FormatMessage>true</FormatMessage> </Syslog> <logLevel>ALERT</logLevel> </MessageLogging>
A common usage of the MessageLogging policy type is to log to a syslog account. When configured for syslog, an API proxy will forward log messages from Apigee to a remote syslog server. You must already have a syslog server available. If not, public log management services, such a Splunk, Sumo Logic, and Loggly, are available. see Configuring third-party log management services.
For example, imagine that you need to log information about each request message that your
API receives from consumer apps. The value 3f509b58
represents a key value
specific to the loggly service. If you have a loggly account, substitute your loggly key. The
log message that is generated will be populated with four values: the organization, API
proxy, and environment name associated with the transaction, along with the value for a query
parameter on the request message.
Syslog over TLS/SSL
<MessageLogging name="LogToSyslog"> <Syslog> <Message>[3f509b58 tag="{organization.name}.{apiproxy.name}.{environment.name}"] Weather request for WOEID {request.queryparam.w}.</Message> <Host>logs-01.loggly.com</Host> <Port>6514</Port> <Protocol>TCP</Protocol> <FormatMessage>true</FormatMessage> <SSLInfo> <Enabled>true</Enabled> </SSLInfo> </Syslog> <logLevel>WARN</logLevel> </MessageLogging>
You can send messages to third-party message logging providers over TLS/SSL by adding the
<SSLInfo>
block.
File rotation: size
<MessageLogging name="LogPolicy"> <File> <Message>This is a test message. Message id : {request.header.messageid}</Message> <FileName>test.log</FileName> <FileRotationOptions rotateFileOnStartup="true"> <FileRotationType>SIZE</FileRotationType> <MaxFileSizeInMB>10</MaxFileSizeInMB> <MaxFilesToRetain>10</MaxFilesToRetain> </FileRotationOptions> </File> <logLevel>ERROR</logLevel> </MessageLogging>
File rotation based on file size.
File rotation: time
<MessageLogging name="LogPolicy"> <File> <Message>This is a test message. Message id : {request.header.messageid}</Message> <FileName>test.log</FileName> <FileRotationOptions rotateFileOnStartup="true"> <FileRotationType>TIME</FileRotationType> <RotationFrequency unit="minute">10</RotationFrequency> <MaxFilesToRetain>10</MaxFilesToRetain> </FileRotationOptions> </File> <logLevel>ERROR</logLevel> </MessageLogging>
File rotation based on time.
File rotation: time & size
<MessageLogging name="LogPolicy"> <File> <Message>This is a test message. Message id : {request.header.messageid}</Message> <FileName>test.log</FileName> <FileRotationOptions rotateFileOnStartup="true"> <FileRotationType>TIME_SIZE</FileRotationType> <MaxFileSizeInMB>10</MaxFileSizeInMB> <MaxFilesToRetain>10</MaxFilesToRetain> <RotationFrequency unit="minute">10</RotationFrequency> </FileRotationOptions> </File> <logLevel>ERROR</logLevel> </MessageLogging>
File rotation based on time and size.
Stream-enabled
<MessageLogging name="LogPolicy"> <File> .... .... </File> <BufferMessage>true</BufferMessage> </MessageLogging>
Stream-enabled message logging
Element reference
Use the elements in the table below to configure the MessageLogging policy type.
To send syslog to Splunk, Sumo Logic, or Loggly, see Configuring third-party log management services.
Field Name | Field Description | |
---|---|---|
Message |
Build the message to be sent to the syslog, combining text with variables to capture the information you want. See the Samples. |
|
Host |
The hostname or IP address of the server where the syslog should be sent. If you don't include this element, the default is localhost. | |
Port |
Port where the syslog is running. If you don't include this element, the default is 514. | |
Protocol |
TCP or UDP (default). While UDP is more performant, the TCP protocol guarantees message log delivery to the syslog server. For sending syslog messages over TLS/SSL, only TCP is supported. | |
FormatMessage |
Optional, but This element lets you control the format of Apigee-generated content prepended to the message. If set to true, the syslog message is prepended by a fixed number of characters, which lets you filter out that information from messages. Here's an example for the fixed format:
The Apigee-generated information includes:
If set to false (default), the message is not prepended with those fixed characters. |
|
PayloadOnly |
This element sets the format of Apigee-generated messages to contain only the body of the syslog message, without the prepended characters specified by FormatMessage. If you don't include this element or leave it empty, the default value is See FormatMessage. |
|
SSLInfo |
Lets you log messages through SSL/TLS. Use with
sub-element If you don't include this element or leave it empty, the default value is false (no TLS/SSL). <SSLInfo> <Enabled>true</Enabled> </SSLInfo> You can configure the <SSLInfo> tag the same as you can on a TargetEndpoint, including enabling two-way TLS/SSL, as described in API proxy configuration reference. Only the TCP protocol is supported. |
|
logLevel |
Optional. Valid values: Set a specific level of information to be included in the message log. If you're using the |
Schemas
Usage notes
When attaching a MessageLogging policy to an API proxy flow, consider placing it in the ProxyEndpoint response, in a special flow called PostClientFlow. The PostClientFlow executes after the response is sent to the requesting client, which ensures that all metrics are available for logging. For details on using PostClientFlow, see API proxy configuration reference.
The PostClientFlow is special in two ways:
- It only executed as part of the response flow.
- It is the only flow executed after the proxy enters the error state.
Because it is executed regardless of whether the proxy succeeded or failed, you can put MessageLogging policies in the PostClientFlow and be guaranteed that they always execute.
The following Trace image shows a MessageLogging policy executing as part of the PostClientFlow, after the DefaultFaultRule executes:
In this example, the Verify API Key policy caused the fault because of an invalid key.
Shown below is the ProxyEndpoint definition that includes the PostClientFlow:
<ProxyEndpoint name="default"> ... <PostClientFlow> <Response> <Step> <Name>Message-Logging-1</Name> </Step> </Response> </PostClientFlow> ... </ProxyEndpoint>
Apigee logs messages as simple text, and you can configure logging to include variables, such as the date and time when the request or response was received, the user identity on the request, the source IP address from which the request was sent, and so on. Apigee logs message asynchronously, meaning that no latency that might be caused by blocking callouts is introduced to your API.
The MessageLogging policy writes logged messages in memory to a buffer. The message logger reads messages from the buffer and then writes to the destination that you configure. Each destination has its own buffer.
If the write rate to the buffer increases beyond the read rate, the buffer overflows and logging will fail. If this happens, you might find a message containing the following in the log file:
Log message size exceeded. Increase the max message size setting
Increase the max.log.message.size.in.kb
property (default value = 128 KB) in the
message-logging.properties
file.
Default values for variables in message template
Default values can be specified for each variable in message template separately. For example,
if the variable request.header.id
cannot be resolved, then its value is replaced
with the value unknown
.
<Message>This is a test message. id = {request.header.id:unknown}</Message>
A common default value can be specified for all the unresolved variables by setting the
defaultVariableValue
attribute on the Message
element:
<Message defaultVariableValue="unknown">This is a test message. id = {request.header.id}</Message>
Configuring third-party log management services
The MessageLogging policy lets you send syslog messages to third-party log management services, such as Splunk, Sumo Logic, and Loggly. If you want to send syslog to one of those services, see that service's documentation to configure the service's host, port, and protocol, then set the Syslog element on this policy accordingly.
See the following documentation for third-party log management configuration:
- Splunk (select the product version)
Also see this Apigee Community post: Log messages into Splunk -
Sumo
Logic
- Also see this Apigee Community post: Setting up Logging with Sumo Logic
- For a complete example using Sumo Logic as the logging service, see the following Apigee Community post. The solution uses a single JavaScript policy to make HTTP POST requests to Sumo Logic HTTP Source Collector: Logging to Sumo Logic using JavaScript and HTTP
- Loggly
When using Loggly,<FormatMessage>true</FormatMessage>
is required in the policy as a child of the<Syslog>
element.
Also see this Apigee Community post for more information about message logging to Loggly: 7 Log Messages into Loggly
Error reference
This section describes the fault codes and error messages that are returned and fault variables that are set by Apigee when this policy triggers an error. This information is important to know if you are developing fault rules to handle faults. To learn more, see What you need to know about policy errors and Handling faults.
Runtime errors
These errors can occur when the policy executes.
Fault code | HTTP status | Cause |
---|---|---|
steps.messagelogging.StepDefinitionExecutionFailed |
500 |
See fault string. |
Deployment errors
These errors can occur when you deploy a proxy containing this policy.
Error name | Cause | Fix |
---|---|---|
InvalidProtocol |
The deployment of the MessageLogging policy can fail with this error if the protocol
specified within the <Protocol> element is not valid. The valid protocols are TCP and UDP.
For sending syslog messages over TLS/SSL, only TCP is supported. |
build |
InvalidPort |
The deployment of the MessageLogging policy can fail with this error if the port number
is not specified within the <Port> element or if it is not valid. The port number must be
an integer greater than zero. |
build |
Fault variables
These variables are set when a runtime error occurs. For more information, see What you need to know about policy errors.
Variables | Where | Example |
---|---|---|
fault.name="fault_name" |
fault_name is the name of the fault, as listed in the Runtime errors table above. The fault name is the last part of the fault code. | fault.name Matches "StepDefinitionExecutionFailed" |
messagelogging.policy_name.failed |
policy_name is the user-specified name of the policy that threw the fault. | messagelogging.ML-LogMessages.failed = true |
Example error response
{ "fault":{ "detail":{ "errorcode":"steps.messagelogging.StepDefinitionExecutionFailed" }, "faultstring":"Execution failed" } }
Example fault rule
<FaultRule name="MessageLogging"> <Step> <Name>ML-LogMessages</Name> <Condition>(fault.name Matches "StepDefinitionExecutionFailed") </Condition> </Step> <Condition>(messagelogging.ML-LogMessages.failed = true) </Condition> </FaultRule>
Flow variables
The following variables are populated on policy failure.
messagelogging.failed
messagelogging.{stepdefinition-name}.failed
Related topics
- Variables exposed by Apigee: Flow variables reference