tag:status.travelbank.com,2005:/historyTravelBank Status - Incident History2024-03-27T19:49:45-07:00TravelBanktag:status.travelbank.com,2005:Incident/202708172024-03-19T10:13:01-07:002024-03-19T10:13:01-07:00QBO Integration Disconnections<p><small>Mar <var data-var='date'>19</var>, <var data-var='time'>10:13</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>09:04</var> PDT</small><br><strong>Investigating</strong> - Due to an infrastructure issue, some QBO integrations may need to be reconnected. Please reconnect your integrations by going to Company Settings > Integrations, then selecting the QBO integration and reconnect.</p>tag:status.travelbank.com,2005:Incident/202845662024-03-18T08:55:18-07:002024-03-18T08:55:18-07:00Network Issues Accessing TravelBank from Mobile<p><small>Mar <var data-var='date'>18</var>, <var data-var='time'>08:55</var> PDT</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>17</var>, <var data-var='time'>09:14</var> PDT</small><br><strong>Monitoring</strong> - A change that was made in our network is causing our mobile apps to have network connectivity. This has been reverted and mobile apps should be working as expected once again.</p><p><small>Mar <var data-var='date'>17</var>, <var data-var='time'>08:52</var> PDT</small><br><strong>Investigating</strong> - We are investigating network issues experienced while using the mobile application. The web application is working as expected.</p>tag:status.travelbank.com,2005:Incident/202325232024-03-17T04:00:57-07:002024-03-17T04:00:57-07:00Scheduled Failover<p><small>Mar <var data-var='date'>17</var>, <var data-var='time'>04:00</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Mar <var data-var='date'>17</var>, <var data-var='time'>01:00</var> PDT</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'>13</var>, <var data-var='time'>03:35</var> PDT</small><br><strong>Scheduled</strong> - We will be undergoing scheduled failover during this time. There should be minimal impact</p>tag:status.travelbank.com,2005:Incident/202713012024-03-15T14:03:38-07:002024-03-15T14:03:38-07:00Delayed Reimbursements<p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>14:03</var> PDT</small><br><strong>Resolved</strong> - All reimbursements have confirmed processed.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>10:08</var> PDT</small><br><strong>Monitoring</strong> - Reimbursements for Friday, March 15th may be delayed a few hours, but they should still be processed by the end of the day.</p>tag:status.travelbank.com,2005:Incident/201780692024-03-10T04:15:56-07:002024-03-10T04:15:56-07:00Scalegrid OS Patch on clusters (Prod, Pre-Prod)<p><small>Mar <var data-var='date'>10</var>, <var data-var='time'>04:15</var> PDT</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Mar <var data-var='date'>10</var>, <var data-var='time'>00:15</var> PST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Mar <var data-var='date'> 6</var>, <var data-var='time'>09:49</var> PST</small><br><strong>Scheduled</strong> - We will be undergoing OS Patching at this time. There should be minimal downtime</p>tag:status.travelbank.com,2005:Incident/201340832024-03-04T12:18:53-08:002024-03-04T12:18:53-08:00Premium Insights Outage<p><small>Mar <var data-var='date'> 4</var>, <var data-var='time'>12:18</var> PST</small><br><strong>Resolved</strong> - Premium Insights is no longer experiencing an outage, but we are actively working to implement a fix that will prevent the cause of the most recent outage to occur again.</p><p><small>Mar <var data-var='date'> 4</var>, <var data-var='time'>11:02</var> PST</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Mar <var data-var='date'> 1</var>, <var data-var='time'>07:54</var> PST</small><br><strong>Investigating</strong> - We are currently experiencing an issue with our Premium Insights feature. We are working to apply a fix</p>tag:status.travelbank.com,2005:Incident/200502752024-03-04T11:03:17-08:002024-03-04T11:03:17-08:00Unable to Create Netsuite Integrations<p><small>Mar <var data-var='date'> 4</var>, <var data-var='time'>11:03</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'>28</var>, <var data-var='time'>10:57</var> PST</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Feb <var data-var='date'>22</var>, <var data-var='time'>09:54</var> PST</small><br><strong>Investigating</strong> - It is not possible to configure new Netsuite integrations at this time. We are working on a fix. Existing integrations are working as expected.</p>tag:status.travelbank.com,2005:Incident/201174622024-02-29T16:05:39-08:002024-02-29T16:05:39-08:00Premium Insights Reports Not Loading<p><small>Feb <var data-var='date'>29</var>, <var data-var='time'>16:05</var> PST</small><br><strong>Resolved</strong> - This has now been resolved.</p><p><small>Feb <var data-var='date'>29</var>, <var data-var='time'>16:00</var> PST</small><br><strong>Identified</strong> - Premium Insights reports are not currently loading. We are working on a resolution</p>tag:status.travelbank.com,2005:Incident/200899762024-02-27T16:33:45-08:002024-03-01T08:53:07-08:00Delays in PDF Generation, Emails<p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>16:33</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>11:19</var> PST</small><br><strong>Update</strong> - Delays in emails and PDF generation should now be resolved. This incident was related to our reporting incident.</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>10:40</var> PST</small><br><strong>Update</strong> - There will be a temporarily delay in emails and PDF generation as we try to restore our reporting database.</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>10:01</var> PST</small><br><strong>Monitoring</strong> - From approximately 5am ET until 1pm ET on Tuesday, February 27th, certain processes were delayed including PDF generation and emails due to an infrastructure event. This has now been resolved. We are working on changes to the system that will reduce the impact to critical operations such as emails when an event like this occurs.</p>tag:status.travelbank.com,2005:Incident/200902002024-02-27T11:19:23-08:002024-02-27T11:19:23-08:00Premium Insights and Core Insights Outage<p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>11:19</var> PST</small><br><strong>Resolved</strong> - Premium and Core Insights should now be working properly. An issue with our reporting infrastructure required us to restart our reporting database.</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>10:31</var> PST</small><br><strong>Investigating</strong> - There is an issue with Premium and Core Insights not loading properly.</p>tag:status.travelbank.com,2005:Incident/200604032024-02-24T17:00:56-08:002024-02-24T17:00:56-08:00Scheduled Maintenance on Ticketing 9am -5pm ET, Saturday February 24<p><small>Feb <var data-var='date'>24</var>, <var data-var='time'>17:00</var> PST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'>24</var>, <var data-var='time'>09:00</var> PST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Feb <var data-var='date'>23</var>, <var data-var='time'>13:02</var> PST</small><br><strong>Update</strong> - We will be undergoing scheduled maintenance during this time.</p><p><small>Feb <var data-var='date'>23</var>, <var data-var='time'>13:01</var> PST</small><br><strong>Scheduled</strong> - Maintenance on Saturday, February 24 may affect ticketing for flight and hotel bookings, but rest assured these bookings will be captured. Please reach out to our travel support team if you experience an extensive delay in receiving a confirmation email for your bookings.</p>tag:status.travelbank.com,2005:Incident/199937692024-02-18T04:15:57-08:002024-02-18T04:15:57-08:00Kubernetes OS upgrade (Prod and warm clusters)<p><small>Feb <var data-var='date'>18</var>, <var data-var='time'>04:15</var> PST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'>18</var>, <var data-var='time'>00:15</var> PST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Feb <var data-var='date'>15</var>, <var data-var='time'>10:03</var> PST</small><br><strong>Scheduled</strong> - We will be undergoing Kubernetes upgrade at this time. There should be minimal downtime.</p>tag:status.travelbank.com,2005:Incident/199413862024-02-11T05:01:12-08:002024-02-11T05:01:12-08:00Kubernetes OS upgrade<p><small>Feb <var data-var='date'>11</var>, <var data-var='time'>05:01</var> PST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'>11</var>, <var data-var='time'>01:01</var> PST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Feb <var data-var='date'> 8</var>, <var data-var='time'>09:56</var> PST</small><br><strong>Scheduled</strong> - We will be performing Kubernetes OS upgrade at this time. There should be minimal downtime.</p>tag:status.travelbank.com,2005:Incident/199324922024-02-11T05:00:56-08:002024-02-11T05:00:56-08:00Database Upgrade<p><small>Feb <var data-var='date'>11</var>, <var data-var='time'>05:00</var> PST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Feb <var data-var='date'>11</var>, <var data-var='time'>01:00</var> PST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Feb <var data-var='date'> 7</var>, <var data-var='time'>09:39</var> PST</small><br><strong>Scheduled</strong> - We will be undergoing database maintenance at this time. There should be minimal downtime</p>tag:status.travelbank.com,2005:Incident/196057772024-01-10T13:04:29-08:002024-01-10T13:04:29-08:00Custom Mileage Rate Updates Not Working<p><small>Jan <var data-var='date'>10</var>, <var data-var='time'>13:04</var> PST</small><br><strong>Resolved</strong> - This issue has been resolved.</p><p><small>Jan <var data-var='date'> 3</var>, <var data-var='time'>13:25</var> PST</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p>tag:status.travelbank.com,2005:Incident/196135142024-01-05T09:20:59-08:002024-01-05T09:20:59-08:00Latency<p><small>Jan <var data-var='date'> 5</var>, <var data-var='time'>09:20</var> PST</small><br><strong>Resolved</strong> - Latency is no longer occurring. We are still investigating root cause.</p><p><small>Jan <var data-var='date'> 4</var>, <var data-var='time'>09:10</var> PST</small><br><strong>Investigating</strong> - We are investigating reports of latency throughout the application</p>tag:status.travelbank.com,2005:Incident/196036642024-01-03T13:24:29-08:002024-01-03T13:24:29-08:00Custom Mileage Rate Updates Not Working<p><small>Jan <var data-var='date'> 3</var>, <var data-var='time'>13:24</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'> 3</var>, <var data-var='time'>08:46</var> PST</small><br><strong>Identified</strong> - The team has identified the issue and is currently working on a solution</p>tag:status.travelbank.com,2005:Incident/195947462024-01-02T16:15:18-08:002024-01-02T16:15:18-08:002024 Mileage Rates Not Available<p><small>Jan <var data-var='date'> 2</var>, <var data-var='time'>16:15</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved. 2024 IRS mileage rates should be added.</p><p><small>Jan <var data-var='date'> 2</var>, <var data-var='time'>09:58</var> PST</small><br><strong>Investigating</strong> - There is an issue with 2024 mileage rates not populating correctly for mileage expenses. We are investigating.</p>tag:status.travelbank.com,2005:Incident/194405952023-12-18T07:57:27-08:002023-12-18T07:57:27-08:00Plaid Chase<p><small>Dec <var data-var='date'>18</var>, <var data-var='time'>07:57</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'>14</var>, <var data-var='time'>10:06</var> PST</small><br><strong>Investigating</strong> - There is an issue with corporate card feeds related Chase Business and Consumer cards. We are aware of the issue and investigating. The impacted feeds have been temporarily disabled while we investigate.</p>tag:status.travelbank.com,2005:Incident/194022272023-12-17T05:00:11-08:002023-12-17T05:00:12-08:00Scheduled Failover<p><small>Dec <var data-var='date'>17</var>, <var data-var='time'>05:00</var> PST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Dec <var data-var='date'>17</var>, <var data-var='time'>01:00</var> PST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Dec <var data-var='date'>11</var>, <var data-var='time'>07:44</var> PST</small><br><strong>Scheduled</strong> - We will be performing a scheduled failover test in our production environment. There should be minimal impact to services.</p>tag:status.travelbank.com,2005:Incident/192525152023-12-09T03:01:08-08:002023-12-09T03:01:08-08:00Infrastructure Update<p><small>Dec <var data-var='date'> 9</var>, <var data-var='time'>03:01</var> PST</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Dec <var data-var='date'> 9</var>, <var data-var='time'>01:00</var> PST</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Nov <var data-var='date'>30</var>, <var data-var='time'>11:53</var> PST</small><br><strong>Scheduled</strong> - We will be performing some updates to our DNS at this time. This should have minimal impact.</p>tag:status.travelbank.com,2005:Incident/191898712023-12-05T09:40:00-08:002023-12-05T09:40:00-08:00Commercial Rewards Card Cardholders Cannot View Their Statements<p><small>Dec <var data-var='date'> 5</var>, <var data-var='time'>09:40</var> PST</small><br><strong>Resolved</strong> - The incident has been resolved. Cardholders can now view their statements.</p><p><small>Nov <var data-var='date'>21</var>, <var data-var='time'>12:07</var> PST</small><br><strong>Identified</strong> - Commercial Rewards Card cardholders cannot view their statements. <br /><br />Until resolved: Only admins have access to statements. To obtain cardholder statements admins should navigate to the Manage Tab > Statements > Cardholder Accounts > choose the cardholder > click "View all" > option to view or download available statements.</p>tag:status.travelbank.com,2005:Incident/192660932023-12-01T15:13:34-08:002023-12-01T15:13:34-08:00Elevated Booking Errors<p><small>Dec <var data-var='date'> 1</var>, <var data-var='time'>15:13</var> PST</small><br><strong>Resolved</strong> - The travel team was able to confirm that booking error rates are within normal range.</p><p><small>Dec <var data-var='date'> 1</var>, <var data-var='time'>12:25</var> PST</small><br><strong>Investigating</strong> - There are reports of increased errors when booking travel. We are investigating.</p>tag:status.travelbank.com,2005:Incident/191322832023-11-16T08:01:58-08:002023-11-16T08:01:58-08:00API Outage<p><small>Nov <var data-var='date'>16</var>, <var data-var='time'>08:01</var> PST</small><br><strong>Resolved</strong> - This incident has been resolved. The system has been stable since recovery.</p><p><small>Nov <var data-var='date'>15</var>, <var data-var='time'>16:57</var> PST</small><br><strong>Update</strong> - The TravelBank system was down for approximately 1 hour and 10 minutes starting at 5:38 pm CT. Systems came back online approximately 6:50 pm CT following restoration procedures by our on-call team. We will provide more updates once we've identified root cause and will share findings and opportunities for improvement.</p><p><small>Nov <var data-var='date'>15</var>, <var data-var='time'>16:50</var> PST</small><br><strong>Monitoring</strong> - An update has been made to our infrastructure to restore functionality to the system.</p><p><small>Nov <var data-var='date'>15</var>, <var data-var='time'>16:39</var> PST</small><br><strong>Investigating</strong> - TravelBank is currently experiencing an API outage. We've identified that is related to one of our core services.</p>tag:status.travelbank.com,2005:Incident/189907222023-11-14T12:30:31-08:002023-11-14T12:30:31-08:00iOS Booking Issues<p><small>Nov <var data-var='date'>14</var>, <var data-var='time'>12:30</var> PST</small><br><strong>Resolved</strong> - This issue has been resolved with the latest iOS update.</p><p><small>Nov <var data-var='date'> 1</var>, <var data-var='time'>08:59</var> PDT</small><br><strong>Identified</strong> - We are currently facing issues booking travel on the iOS app. The issue has been identified and we are working to resolve the issue as soon as possible.</p>