By type
WARN [2026-09-25 07:40:58,703] com.datastax.driver.core.utils.UUIDs: PID returned through native call was 0, JNR versions incompatible? Falling back to JMX.
INFO [2026-09-25 07:41:00,700] [main] o.e.j.u.log - Logging initialized @2648ms to org.eclipse.jetty.util.log.Slf4jLog
INFO [2026-09-25 07:41:00,797] [main] i.d.s.DefaultServerFactory - Registering jersey handler with root path prefix: /
INFO [2026-09-25 07:41:00,799] [main] i.d.s.DefaultServerFactory - Registering admin handler with root path prefix: /
INFO [2026-09-25 07:41:00,800] [main] i.d.a.AssetsBundle - Registering AssetBundle with name: assets for path /webui/*
INFO [2026-09-25 07:41:00,974] [main] o.a.c.b.FluentPropertyBeanIntrospector - Error when creating PropertyDescriptor for public void org.apache.shiro.session.mgt.AbstractNativeSessionManager.setTimeout(org.apache.shiro.session.mgt.SessionKey,long) throws org.apache.shiro.session.InvalidSessionException! Ignoring this property.
INFO [2026-09-25 07:41:00,974] [main] o.a.c.b.FluentPropertyBeanIntrospector - Error when creating PropertyDescriptor for public void org.apache.shiro.session.mgt.AbstractNativeSessionManager.setAttribute(org.apache.shiro.session.mgt.SessionKey,java.lang.Object,java.lang.Object) throws org.apache.shiro.session.InvalidSessionException! Ignoring this property.
INFO [2026-09-25 07:41:00,992] [main] o.a.s.c.ReflectionBuilder - An instance with name 'authc' already exists. Redefining this object as a new instance of type org.apache.shiro.web.filter.authc.PassThruAuthenticationFilter
INFO [2026-09-25 07:41:00,999] [main] o.a.s.c.ReflectionBuilder - An instance with name 'rest' already exists. Redefining this object as a new instance of type io.cassandrareaper.resources.auth.RestPermissionsFilter
INFO [2026-09-25 07:41:01,022] [main] i.c.ReaperApplication - initializing runner thread pool with 15 threads and 2 repair runners
INFO [2026-09-25 07:41:01,022] [main] i.c.ReaperApplication - initializing storage of type: cassandra
INFO [2026-09-25 07:41:01,023] [main] i.c.s.InitializeStorage - Initializing the database and performing schema migrations
INFO [2026-09-25 07:41:01,037] [main] c.d.d.core - DataStax Java driver 3.11.5 for Apache Cassandra
INFO [2026-09-25 07:41:01,041] [main] c.d.d.c.GuavaCompatibility - Detected Guava >= 19 in the classpath, using modern compatibility layer
INFO [2026-09-25 07:41:01,163] [main] c.d.d.c.Native - Could not load JNR C Library, native system calls through this library will not be available (set this logger level to DEBUG to see the full stack trace).
INFO [2026-09-25 07:41:01,163] [main] c.d.d.c.ClockFactory - Using java.lang.System clock to generate timestamps.
INFO [2026-09-25 07:41:01,330] [main] c.d.d.c.NettyUtil - Did not find Netty's native epoll transport in the classpath, defaulting to NIO.
INFO [2026-09-25 07:41:02,228] [main] c.d.d.c.p.DCAwareRoundRobinPolicy - Using provided data-center name 'dc1' for DCAwareRoundRobinPolicy
INFO [2026-09-25 07:41:02,230] [main] c.d.d.c.Cluster - New Cassandra host cassandra-dc1-service/10.233.69.92:9042 added
INFO [2026-09-25 07:41:02,230] [main] c.d.d.c.Cluster - New Cassandra host cassandra-dc1-service/10.233.72.155:9042 added
INFO [2026-09-25 07:41:02,230] [main] c.d.d.c.Cluster - New Cassandra host cassandra-dc1-service/10.233.66.208:9042 added
INFO [2026-09-25 07:41:03,191] [main] o.c.c.m.MigrationRepository - Found 18 migration scripts
INFO [2026-09-25 07:41:03,191] [main] i.c.s.c.MigrationManager - Keyspace reaper_db already at schema version 33
INFO [2026-09-25 07:41:03,626] [main] i.c.ReaperApplication - no management connection factory given in context, creating default
INFO [2026-09-25 07:41:03,626] [main] i.c.ReaperApplication - HTTP management connection config not set, or set disabled. Creating JMX connection factory instead
INFO [2026-09-25 07:41:03,630] [main] i.c.m.j.JmxManagementConnectionFactory - Initializing JMX seed list for all clusters...
INFO [2026-09-25 07:41:03,647] [main] i.c.m.j.JmxManagementConnectionFactory - Initialized JMX seed list for all clusters.
INFO [2026-09-25 07:41:03,659] [main] i.c.ReaperApplication - creating and registering health checks
INFO [2026-09-25 07:41:03,659] [main] i.c.ReaperApplication - creating resources and registering endpoints
INFO [2026-09-25 07:41:05,057] [main] i.c.s.SchedulingManager - Starting new SchedulingManager instance
INFO [2026-09-25 07:41:05,061] [main] i.c.s.AutoSchedulingManager - Starting new AutoSchedulingManager instance. First check in 15000ms. Subsequent polls every 600000ms
INFO [2026-09-25 07:41:05,087] [main] i.c.ReaperApplication - resuming pending repair runs
INFO [2026-09-25 07:41:05,107] [ReaperApplication-scheduler] i.c.ReaperApplication - Purged 0 repair runs from history
INFO [2026-09-25 07:41:05,107] [main] i.c.ReaperApplication - Initialization complete!
WARN [2026-09-25 07:41:05,107] [main] i.c.ReaperApplication - Reaper is ready to get things done!
INFO [2026-09-25 07:41:05,109] [main] i.d.s.ServerFactory - Starting cassandra-reaper
_________ .___ __________
\_ ___ \_____ ______ ___________ ____ __| _/___________ \______ \ ____ _____ ______ ___________
/ \ \/\__ \ / ___// ___/\__ \ / \ / __ |\_ __ \__ \ | _// __ \\__ \ \____ \_/ __ \_ __ \
\ \____/ __ \_\___ \ \___ \ / __ \| | \/ /_/ | | | \// __ \_ | | \ ___/ / __ \| |_> > ___/| | \/
\______ (____ /____ >____ >(____ /___| /\____ | |__| (____ / |____|_ /\___ >____ / __/ \___ >__|
\/ \/ \/ \/ \/ \/ \/ \/ \/ \/ \/|__| \/
INFO [2026-09-25 07:41:05,279] [main] o.e.j.s.SetUIDListener - Opened application@7159d601{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
INFO [2026-09-25 07:41:05,279] [main] o.e.j.s.SetUIDListener - Opened admin@70b33256{HTTP/1.1, (http/1.1)}{0.0.0.0:8081}
INFO [2026-09-25 07:41:05,283] [main] o.e.j.s.Server - jetty-9.4.53.v20231009; built: 2023-10-09T12:29:09.265Z; git: 27bde00a0b95a1d5bbee0eae7984f891d2d0f8c9; jvm 11.0.26+4-LTS
INFO [2026-09-25 07:41:05,309] [main] o.e.j.s.session - DefaultSessionIdManager workerName=node0
INFO [2026-09-25 07:41:05,309] [main] o.e.j.s.session - No SessionScavenger set, using defaults
INFO [2026-09-25 07:41:05,311] [main] o.e.j.s.session - node0 Scavenging every 660000ms
INFO [2026-09-25 07:41:06,230] [main] i.d.j.DropwizardResourceConfig - The following paths were found for the configured resources:
GET /cluster (io.cassandrareaper.resources.ClusterResource)
POST /cluster (io.cassandrareaper.resources.ClusterResource)
POST /cluster/auth (io.cassandrareaper.resources.ClusterResource)
PUT /cluster/auth/{cluster_name} (io.cassandrareaper.resources.ClusterResource)
DELETE /cluster/{cluster_name} (io.cassandrareaper.resources.ClusterResource)
GET /cluster/{cluster_name} (io.cassandrareaper.resources.ClusterResource)
PUT /cluster/{cluster_name} (io.cassandrareaper.resources.ClusterResource)
GET /cluster/{cluster_name}/tables (io.cassandrareaper.resources.ClusterResource)
GET /crypto/encrypt/{text} (io.cassandrareaper.resources.CryptoResource)
GET /diag_event/sse_listen/{id} (io.cassandrareaper.resources.DiagEventSseResource)
GET /diag_event/subscription (io.cassandrareaper.resources.DiagEventSubscriptionResource)
POST /diag_event/subscription (io.cassandrareaper.resources.DiagEventSubscriptionResource)
GET /diag_event/subscription/adhoc (io.cassandrareaper.resources.DiagEventSubscriptionResource)
GET /diag_event/subscription/pollers (io.cassandrareaper.resources.DiagEventSubscriptionResource)
DELETE /diag_event/subscription/{id} (io.cassandrareaper.resources.DiagEventSubscriptionResource)
GET /diag_event/subscription/{id} (io.cassandrareaper.resources.DiagEventSubscriptionResource)
GET /jwt (io.cassandrareaper.resources.auth.ShiroJwtProvider)
POST /login (io.cassandrareaper.resources.auth.LoginResource)
POST /logout (io.cassandrareaper.resources.auth.LoginResource)
GET /node/clientRequestLatencies/{clusterName}/{host} (io.cassandrareaper.resources.NodeStatsResource)
GET /node/compactions/{clusterName}/{host} (io.cassandrareaper.resources.NodeStatsResource)
GET /node/dropped/{clusterName}/{host} (io.cassandrareaper.resources.NodeStatsResource)
GET /node/streams/{clusterName}/{host} (io.cassandrareaper.resources.NodeStatsResource)
GET /node/tokens/{clusterName}/{host} (io.cassandrareaper.resources.NodeStatsResource)
GET /node/tpstats/{clusterName}/{host} (io.cassandrareaper.resources.NodeStatsResource)
GET /ping (io.cassandrareaper.resources.PingResource)
HEAD /ping (io.cassandrareaper.resources.PingResource)
GET /reaper/datacenterAvailability (io.cassandrareaper.resources.ReaperResource)
GET /repair_run (io.cassandrareaper.resources.RepairRunResource)
POST /repair_run (io.cassandrareaper.resources.RepairRunResource)
GET /repair_run/cluster/{clusterName} (io.cassandrareaper.resources.RepairRunResource)
POST /repair_run/purge (io.cassandrareaper.resources.RepairRunResource)
DELETE /repair_run/{id} (io.cassandrareaper.resources.RepairRunResource)
GET /repair_run/{id} (io.cassandrareaper.resources.RepairRunResource)
PUT /repair_run/{id}/intensity/{intensity} (io.cassandrareaper.resources.RepairRunResource)
GET /repair_run/{id}/segments (io.cassandrareaper.resources.RepairRunResource)
POST /repair_run/{id}/segments/abort/{segment_id} (io.cassandrareaper.resources.RepairRunResource)
PUT /repair_run/{id}/state/{state} (io.cassandrareaper.resources.RepairRunResource)
GET /repair_schedule (io.cassandrareaper.resources.RepairScheduleResource)
POST /repair_schedule (io.cassandrareaper.resources.RepairScheduleResource)
GET /repair_schedule/cluster/{cluster_name} (io.cassandrareaper.resources.RepairScheduleResource)
POST /repair_schedule/start/{id} (io.cassandrareaper.resources.RepairScheduleResource)
GET /repair_schedule/{clusterName}/{id}/percent_repaired (io.cassandrareaper.resources.RepairScheduleResource)
DELETE /repair_schedule/{id} (io.cassandrareaper.resources.RepairScheduleResource)
GET /repair_schedule/{id} (io.cassandrareaper.resources.RepairScheduleResource)
PATCH /repair_schedule/{id} (io.cassandrareaper.resources.RepairScheduleResource)
PUT /repair_schedule/{id} (io.cassandrareaper.resources.RepairScheduleResource)
GET /snapshot/cluster/{clusterName} (io.cassandrareaper.resources.SnapshotResource)
POST /snapshot/cluster/{clusterName} (io.cassandrareaper.resources.SnapshotResource)
GET /snapshot/cluster/{clusterName}/{host} (io.cassandrareaper.resources.SnapshotResource)
POST /snapshot/cluster/{clusterName}/{host} (io.cassandrareaper.resources.SnapshotResource)
DELETE /snapshot/cluster/{clusterName}/{host}/{snapshotName} (io.cassandrareaper.resources.SnapshotResource)
DELETE /snapshot/cluster/{clusterName}/{snapshotName} (io.cassandrareaper.resources.SnapshotResource)
INFO [2026-09-25 07:41:06,233] [main] o.e.j.s.h.ContextHandler - Started i.d.j.MutableServletContextHandler@299eca90{/,null,AVAILABLE}
INFO [2026-09-25 07:41:06,238] [main] i.d.s.AdminEnvironment - tasks =
POST /tasks/log-level (io.dropwizard.servlets.tasks.LogConfigurationTask)
POST /tasks/gc (io.dropwizard.servlets.tasks.GarbageCollectionTask)
INFO [2026-09-25 07:41:06,240] [main] o.e.j.s.h.ContextHandler - Started i.d.j.MutableServletContextHandler@6d2db15b{/,null,AVAILABLE}
INFO [2026-09-25 07:41:06,251] [main] o.e.j.s.AbstractConnector - Started application@7159d601{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
INFO [2026-09-25 07:41:06,255] [main] o.e.j.s.AbstractConnector - Started admin@70b33256{HTTP/1.1, (http/1.1)}{0.0.0.0:8081}
INFO [2026-09-25 07:41:06,255] [main] o.e.j.s.Server - Started @8205ms
WARN [2026-09-25 07:41:45,683] [dw-admin-65] o.e.j.s.HttpChannel - /healthcheck
java.lang.NullPointerException: null
at com.codahale.metrics.health.HealthCheck.execute(HealthCheck.java:378)
at com.codahale.metrics.health.HealthCheckRegistry.runHealthChecks(HealthCheckRegistry.java:184)
at com.codahale.metrics.servlets.HealthCheckServlet.runHealthChecks(HealthCheckServlet.java:177)
at com.codahale.metrics.servlets.HealthCheckServlet.doGet(HealthCheckServlet.java:146)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:497)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:584)
at com.codahale.metrics.servlets.AdminServlet.service(AdminServlet.java:153)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:584)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:799)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1656)
at io.dropwizard.jersey.filter.AllowedMethodsFilter.handle(AllowedMethodsFilter.java:47)
at io.dropwizard.jersey.filter.AllowedMethodsFilter.doFilter(AllowedMethodsFilter.java:41)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1626)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:552)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1440)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:505)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1355)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at io.dropwizard.jetty.RoutingHandler.handle(RoutingHandler.java:52)
at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:772)
at io.dropwizard.jetty.ZipExceptionHandlingGzipHandler.handle(ZipExceptionHandlingGzipHandler.java:26)
at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:181)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:487)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:732)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:479)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:277)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034)
at java.base/java.lang.Thread.run(Thread.java:829)
INFO [2026-09-25 07:42:15,580] [dw-50] o.a.s.s.m.AbstractValidatingSessionManager - Enabling session validation scheduler...
INFO [2026-09-25 07:42:16,014] [dw-49 - PUT /cluster/cassandra?seedHost=cassandra-seed-service] i.c.r.ClusterResource - PUT addOrUpdateCluster called with: cluster_name = cassandra, seedHost = cassandra-seed-service
INFO [2026-09-25 07:42:17,878] [dw-49 - PUT /cluster/cassandra?seedHost=cassandra-seed-service] i.c.r.ClusterResource - creating new cluster based on given seed host: cassandra
WARN [2026-09-25 07:42:18,112] [dw-49 - PUT /cluster/cassandra?seedHost=cassandra-seed-service] i.c.s.ClusterRepairScheduler - No tables found for keyspace sdctitan in cluster cassandra. No repair will be scheduled for this keyspace.
INFO [2026-09-25 07:42:18,415] [dw-49 - PUT /cluster/cassandra?seedHost=cassandra-seed-service] i.c.s.ClusterRepairScheduler - Scheduled repair created: RepairSchedule[a26ca810-b8b4-11f1-a726-ebd0550bac6c]
INFO [2026-09-25 07:42:18,499] [dw-49 - PUT /cluster/cassandra?seedHost=cassandra-seed-service] i.c.s.ClusterRepairScheduler - Scheduled repair created: RepairSchedule[a27bea50-b8b4-11f1-a726-ebd0550bac6c]
INFO [2026-09-25 07:42:18,590] [dw-49 - PUT /cluster/cassandra?seedHost=cassandra-seed-service] i.c.s.ClusterRepairScheduler - Scheduled repair created: RepairSchedule[a28a1b20-b8b4-11f1-a726-ebd0550bac6c]
INFO [2026-09-25 07:42:18,688] [dw-49 - PUT /cluster/cassandra?seedHost=cassandra-seed-service] i.c.s.ClusterRepairScheduler - Scheduled repair created: RepairSchedule[a297d6c0-b8b4-11f1-a726-ebd0550bac6c]
INFO [2026-09-25 07:42:18,799] [dw-49 - PUT /cluster/cassandra?seedHost=cassandra-seed-service] i.c.s.ClusterRepairScheduler - Scheduled repair created: RepairSchedule[a2a93be0-b8b4-11f1-a726-ebd0550bac6c]
INFO [2026-09-25 07:42:18,925] [dw-49 - PUT /cluster/cassandra?seedHost=cassandra-seed-service] i.c.s.ClusterRepairScheduler - Scheduled repair created: RepairSchedule[a2bc4eb0-b8b4-11f1-a726-ebd0550bac6c]
INFO [2026-09-25 07:42:19,026] [dw-49 - PUT /cluster/cassandra?seedHost=cassandra-seed-service] i.c.s.ClusterRepairScheduler - Scheduled repair created: RepairSchedule[a2cc0620-b8b4-11f1-a726-ebd0550bac6c]
INFO [2026-09-25 07:43:06,630] [SchedulingManagerTimer] i.c.s.SchedulingManager - repair unit 'a26a3710-b8b4-11f1-a726-ebd0550bac6c' should be repaired based on RepairSchedule with id 'a26ca810-b8b4-11f1-a726-ebd0550bac6c'
INFO [2026-09-25 07:43:06,691] [SchedulingManagerTimer] i.c.s.RepairManager - Starting a run with id #bf33c0a0-b8b4-11f1-a726-ebd0550bac6c with current state 'NOT_STARTED'
INFO [2026-09-25 07:43:06,831] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Attempting to run new segment...
INFO [2026-09-25 07:43:06,949] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Failed grabbing metrics from 10.233.69.92
java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: A metric named CompactionProxy.pool.size already exists
at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
at io.cassandrareaper.service.RepairRunner.nodesReadyForNewRepair(RepairRunner.java:630)
at io.cassandrareaper.service.RepairRunner.startNextSegment(RepairRunner.java:552)
at io.cassandrareaper.service.RepairRunner.run(RepairRunner.java:251)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:125)
at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57)
at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:78)
at com.codahale.metrics.InstrumentedScheduledExecutorService$InstrumentedRunnable.run(InstrumentedScheduledExecutorService.java:241)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at com.codahale.metrics.InstrumentedThreadFactory$InstrumentedRunnable.run(InstrumentedThreadFactory.java:66)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: A metric named CompactionProxy.pool.size already exists
at com.codahale.metrics.MetricRegistry.register(MetricRegistry.java:168)
at com.codahale.metrics.MetricRegistry.registerGauge(MetricRegistry.java:88)
at com.codahale.metrics.InstrumentedExecutorService.(InstrumentedExecutorService.java:61)
at io.cassandrareaper.management.CompactionProxy.(CompactionProxy.java:50)
at io.cassandrareaper.management.CompactionProxy.create(CompactionProxy.java:56)
at io.cassandrareaper.management.ClusterFacade.listCompactionStatsDirect(ClusterFacade.java:573)
at io.cassandrareaper.management.ClusterFacade.listActiveCompactions(ClusterFacade.java:548)
at io.cassandrareaper.service.RepairRunner.lambda$getNodeMetrics$18(RepairRunner.java:599)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
... 1 common frames omitted
INFO [2026-09-25 07:43:06,985] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Postponed repair segment bf340ec0-b8b4-11f1-a726-ebd0550bac6c on repair run with id bf33c0a0-b8b4-11f1-a726-ebd0550bac6c because we couldn't get hosts metrics on 10.233.69.92
INFO [2026-09-25 07:43:07,063] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Next segment to run : bf33e7b0-b8b4-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:43:15,182] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf33e7b0-b8b4-11f1-a726-ebd0550bac6c] i.c.m.ICassandraManagementProxy - Triggering repair for keyspace "reaper_db" on host 10.233.69.92, with repair parallelism dc_parallel, in cluster with Cassandra version '4.1.8' (can use DATACENTER_AWARE 'true'), for column families: [cluster, diagnostic_event_subscription, leader, node_metrics_v1, node_metrics_v3, node_operations, percent_repaired_by_schedule, repair_run, repair_run_by_cluster, repair_run_by_cluster_v2, repair_run_by_unit, repair_schedule_by_cluster_and_keyspace, repair_schedule_v1, repair_unit_v1, running_reapers, running_repairs, schema_migration, schema_migration_leader, snapshot]
INFO [2026-09-25 07:43:15,182] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf33e7b0-b8b4-11f1-a726-ebd0550bac6c] i.c.m.ICassandraManagementProxy - Triggering repair for ranges null
INFO [2026-09-25 07:43:15,227] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf33e7b0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Triggered repair of segment bf33e7b0-b8b4-11f1-a726-ebd0550bac6c via host 10.233.69.92
INFO [2026-09-25 07:43:15,227] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf33e7b0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.SegmentRunner - Repair for segment bf33e7b0-b8b4-11f1-a726-ebd0550bac6c started, status wait will timeout in 18000000 millis
INFO [2026-09-25 07:43:20,029] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf33e7b0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.SegmentRunner - Repair command 1 on segment bf33e7b0-b8b4-11f1-a726-ebd0550bac6c returned with state DONE
INFO [2026-09-25 07:43:20,069] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf33e7b0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.c.CassandraConcurrencyDao - Trying to release lead on segment bf33c0a0-b8b4-11f1-a726-ebd0550bac6c for instance 72f551e0-b8b4-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:43:20,070] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf33e7b0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.c.CassandraConcurrencyDao - Released lead on segment bf33c0a0-b8b4-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:43:45,302] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Attempting to run new segment...
INFO [2026-09-25 07:43:45,349] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Next segment to run : bf340ec0-b8b4-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:43:49,618] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf340ec0-b8b4-11f1-a726-ebd0550bac6c] i.c.m.ICassandraManagementProxy - Triggering repair for keyspace "reaper_db" on host 10.233.69.92, with repair parallelism dc_parallel, in cluster with Cassandra version '4.1.8' (can use DATACENTER_AWARE 'true'), for column families: [cluster, diagnostic_event_subscription, leader, node_metrics_v1, node_metrics_v3, node_operations, percent_repaired_by_schedule, repair_run, repair_run_by_cluster, repair_run_by_cluster_v2, repair_run_by_unit, repair_schedule_by_cluster_and_keyspace, repair_schedule_v1, repair_unit_v1, running_reapers, running_repairs, schema_migration, schema_migration_leader, snapshot]
INFO [2026-09-25 07:43:49,618] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf340ec0-b8b4-11f1-a726-ebd0550bac6c] i.c.m.ICassandraManagementProxy - Triggering repair for ranges null
INFO [2026-09-25 07:43:49,640] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf340ec0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Triggered repair of segment bf340ec0-b8b4-11f1-a726-ebd0550bac6c via host 10.233.69.92
INFO [2026-09-25 07:43:49,640] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf340ec0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.SegmentRunner - Repair for segment bf340ec0-b8b4-11f1-a726-ebd0550bac6c started, status wait will timeout in 18000000 millis
INFO [2026-09-25 07:43:51,298] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf340ec0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.SegmentRunner - Repair command 2 on segment bf340ec0-b8b4-11f1-a726-ebd0550bac6c returned with state DONE
INFO [2026-09-25 07:43:51,322] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf340ec0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.c.CassandraConcurrencyDao - Trying to release lead on segment bf33c0a0-b8b4-11f1-a726-ebd0550bac6c for instance 72f551e0-b8b4-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:43:51,322] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf340ec0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.c.CassandraConcurrencyDao - Released lead on segment bf33c0a0-b8b4-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:44:06,612] [SchedulingManagerTimer] i.c.s.SchedulingManager - repair unit 'a26a3710-b8b4-11f1-a726-ebd0550bac6c' should be repaired based on RepairSchedule with id 'a26ca810-b8b4-11f1-a726-ebd0550bac6c'
INFO [2026-09-25 07:44:06,669] [SchedulingManagerTimer] i.c.s.SchedulingManager - there is repair (id bf33c0a0-b8b4-11f1-a726-ebd0550bac6c) in state 'RUNNING' for repair unit 'a26a3710-b8b4-11f1-a726-ebd0550bac6c', postponing current schedule trigger until next scheduling
INFO [2026-09-25 07:44:19,534] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Attempting to run new segment...
INFO [2026-09-25 07:44:19,610] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Next segment to run : bf340ec1-b8b4-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:44:29,422] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf340ec1-b8b4-11f1-a726-ebd0550bac6c] i.c.m.ICassandraManagementProxy - Triggering repair for keyspace "reaper_db" on host 10.233.72.155, with repair parallelism dc_parallel, in cluster with Cassandra version '4.1.8' (can use DATACENTER_AWARE 'true'), for column families: [cluster, diagnostic_event_subscription, leader, node_metrics_v1, node_metrics_v3, node_operations, percent_repaired_by_schedule, repair_run, repair_run_by_cluster, repair_run_by_cluster_v2, repair_run_by_unit, repair_schedule_by_cluster_and_keyspace, repair_schedule_v1, repair_unit_v1, running_reapers, running_repairs, schema_migration, schema_migration_leader, snapshot]
INFO [2026-09-25 07:44:29,423] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf340ec1-b8b4-11f1-a726-ebd0550bac6c] i.c.m.ICassandraManagementProxy - Triggering repair for ranges null
INFO [2026-09-25 07:44:29,481] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf340ec1-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Triggered repair of segment bf340ec1-b8b4-11f1-a726-ebd0550bac6c via host 10.233.72.155
INFO [2026-09-25 07:44:29,481] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf340ec1-b8b4-11f1-a726-ebd0550bac6c] i.c.s.SegmentRunner - Repair for segment bf340ec1-b8b4-11f1-a726-ebd0550bac6c started, status wait will timeout in 18000000 millis
INFO [2026-09-25 07:44:36,237] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf340ec1-b8b4-11f1-a726-ebd0550bac6c] i.c.s.SegmentRunner - Repair command 1 on segment bf340ec1-b8b4-11f1-a726-ebd0550bac6c returned with state DONE
INFO [2026-09-25 07:44:36,261] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf340ec1-b8b4-11f1-a726-ebd0550bac6c] i.c.s.c.CassandraConcurrencyDao - Trying to release lead on segment bf33c0a0-b8b4-11f1-a726-ebd0550bac6c for instance 72f551e0-b8b4-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:44:36,261] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c:bf340ec1-b8b4-11f1-a726-ebd0550bac6c] i.c.s.c.CassandraConcurrencyDao - Released lead on segment bf33c0a0-b8b4-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:44:59,358] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Attempting to run new segment...
INFO [2026-09-25 07:44:59,368] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - All nodes are busy or have too many pending compactions for the remaining candidate segments.
INFO [2026-09-25 07:44:59,381] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - All nodes are busy or have too many pending compactions for the remaining candidate segments.
INFO [2026-09-25 07:44:59,392] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Repair amount done 3.0
INFO [2026-09-25 07:44:59,392] [cassandra:bf33c0a0-b8b4-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Repairs for repair run #bf33c0a0-b8b4-11f1-a726-ebd0550bac6c done
INFO [2026-09-25 07:51:20,225] [AutoSchedulingManagerTimer] i.c.s.ClusterRepairScheduler - Scheduled repair created: RepairSchedule[e5618180-b8b5-11f1-a726-ebd0550bac6c]
INFO [2026-09-25 07:51:20,303] [AutoSchedulingManagerTimer] i.c.s.ClusterRepairScheduler - Scheduled repair created: RepairSchedule[e56d1a40-b8b5-11f1-a726-ebd0550bac6c]
INFO [2026-09-25 07:51:20,357] [AutoSchedulingManagerTimer] i.c.s.ClusterRepairScheduler - Scheduled repair created: RepairSchedule[e5757eb0-b8b5-11f1-a726-ebd0550bac6c]
INFO [2026-09-25 07:57:06,593] [SchedulingManagerTimer] i.c.s.SchedulingManager - repair unit 'e5607010-b8b5-11f1-a726-ebd0550bac6c' should be repaired based on RepairSchedule with id 'e5618180-b8b5-11f1-a726-ebd0550bac6c'
INFO [2026-09-25 07:57:06,669] [SchedulingManagerTimer] i.c.s.RepairManager - Starting a run with id #b3e0f860-b8b6-11f1-a726-ebd0550bac6c with current state 'NOT_STARTED'
INFO [2026-09-25 07:57:06,698] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Attempting to run new segment...
INFO [2026-09-25 07:57:06,740] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Next segment to run : b3e11f70-b8b6-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:57:15,297] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f70-b8b6-11f1-a726-ebd0550bac6c] i.c.m.ICassandraManagementProxy - Triggering repair for keyspace "sdctitan" on host 10.233.66.208, with repair parallelism dc_parallel, in cluster with Cassandra version '4.1.8' (can use DATACENTER_AWARE 'true'), for column families: [edgestore, edgestore_lock_, graphindex, graphindex_lock_, janusgraph_ids, system_properties, system_properties_lock_, systemlog, txlog]
INFO [2026-09-25 07:57:15,298] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f70-b8b6-11f1-a726-ebd0550bac6c] i.c.m.ICassandraManagementProxy - Triggering repair for ranges null
INFO [2026-09-25 07:57:15,333] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f70-b8b6-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Triggered repair of segment b3e11f70-b8b6-11f1-a726-ebd0550bac6c via host 10.233.66.208
INFO [2026-09-25 07:57:15,333] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f70-b8b6-11f1-a726-ebd0550bac6c] i.c.s.SegmentRunner - Repair for segment b3e11f70-b8b6-11f1-a726-ebd0550bac6c started, status wait will timeout in 18000000 millis
INFO [2026-09-25 07:57:18,569] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f70-b8b6-11f1-a726-ebd0550bac6c] i.c.s.SegmentRunner - Repair command 1 on segment b3e11f70-b8b6-11f1-a726-ebd0550bac6c returned with state DONE
INFO [2026-09-25 07:57:18,601] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f70-b8b6-11f1-a726-ebd0550bac6c] i.c.s.c.CassandraConcurrencyDao - Trying to release lead on segment b3e0f860-b8b6-11f1-a726-ebd0550bac6c for instance 72f551e0-b8b4-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:57:18,601] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f70-b8b6-11f1-a726-ebd0550bac6c] i.c.s.c.CassandraConcurrencyDao - Released lead on segment b3e0f860-b8b6-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:57:45,274] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Attempting to run new segment...
INFO [2026-09-25 07:57:45,304] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Next segment to run : b3e11f72-b8b6-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:57:48,153] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f72-b8b6-11f1-a726-ebd0550bac6c] i.c.m.ICassandraManagementProxy - Triggering repair for keyspace "sdctitan" on host 10.233.72.155, with repair parallelism dc_parallel, in cluster with Cassandra version '4.1.8' (can use DATACENTER_AWARE 'true'), for column families: [edgestore, edgestore_lock_, graphindex, graphindex_lock_, janusgraph_ids, system_properties, system_properties_lock_, systemlog, txlog]
INFO [2026-09-25 07:57:48,153] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f72-b8b6-11f1-a726-ebd0550bac6c] i.c.m.ICassandraManagementProxy - Triggering repair for ranges null
INFO [2026-09-25 07:57:48,186] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f72-b8b6-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Triggered repair of segment b3e11f72-b8b6-11f1-a726-ebd0550bac6c via host 10.233.72.155
INFO [2026-09-25 07:57:48,186] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f72-b8b6-11f1-a726-ebd0550bac6c] i.c.s.SegmentRunner - Repair for segment b3e11f72-b8b6-11f1-a726-ebd0550bac6c started, status wait will timeout in 18000000 millis
INFO [2026-09-25 07:57:49,106] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f72-b8b6-11f1-a726-ebd0550bac6c] i.c.s.SegmentRunner - Repair command 2 on segment b3e11f72-b8b6-11f1-a726-ebd0550bac6c returned with state DONE
INFO [2026-09-25 07:57:49,125] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f72-b8b6-11f1-a726-ebd0550bac6c] i.c.s.c.CassandraConcurrencyDao - Trying to release lead on segment b3e0f860-b8b6-11f1-a726-ebd0550bac6c for instance 72f551e0-b8b4-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:57:49,125] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f72-b8b6-11f1-a726-ebd0550bac6c] i.c.s.c.CassandraConcurrencyDao - Released lead on segment b3e0f860-b8b6-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:58:18,140] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Attempting to run new segment...
INFO [2026-09-25 07:58:18,180] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Next segment to run : b3e11f71-b8b6-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:58:20,637] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f71-b8b6-11f1-a726-ebd0550bac6c] i.c.m.ICassandraManagementProxy - Triggering repair for keyspace "sdctitan" on host 10.233.69.92, with repair parallelism dc_parallel, in cluster with Cassandra version '4.1.8' (can use DATACENTER_AWARE 'true'), for column families: [edgestore, edgestore_lock_, graphindex, graphindex_lock_, janusgraph_ids, system_properties, system_properties_lock_, systemlog, txlog]
INFO [2026-09-25 07:58:20,637] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f71-b8b6-11f1-a726-ebd0550bac6c] i.c.m.ICassandraManagementProxy - Triggering repair for ranges null
INFO [2026-09-25 07:58:20,656] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f71-b8b6-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Triggered repair of segment b3e11f71-b8b6-11f1-a726-ebd0550bac6c via host 10.233.69.92
INFO [2026-09-25 07:58:20,656] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f71-b8b6-11f1-a726-ebd0550bac6c] i.c.s.SegmentRunner - Repair for segment b3e11f71-b8b6-11f1-a726-ebd0550bac6c started, status wait will timeout in 18000000 millis
INFO [2026-09-25 07:58:21,240] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f71-b8b6-11f1-a726-ebd0550bac6c] i.c.s.SegmentRunner - Repair command 3 on segment b3e11f71-b8b6-11f1-a726-ebd0550bac6c returned with state DONE
INFO [2026-09-25 07:58:21,259] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f71-b8b6-11f1-a726-ebd0550bac6c] i.c.s.c.CassandraConcurrencyDao - Trying to release lead on segment b3e0f860-b8b6-11f1-a726-ebd0550bac6c for instance 72f551e0-b8b4-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:58:21,260] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c:b3e11f71-b8b6-11f1-a726-ebd0550bac6c] i.c.s.c.CassandraConcurrencyDao - Released lead on segment b3e0f860-b8b6-11f1-a726-ebd0550bac6c
INFO [2026-09-25 07:58:50,617] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Attempting to run new segment...
INFO [2026-09-25 07:58:50,629] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - All nodes are busy or have too many pending compactions for the remaining candidate segments.
INFO [2026-09-25 07:58:50,641] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - All nodes are busy or have too many pending compactions for the remaining candidate segments.
INFO [2026-09-25 07:58:50,649] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Repair amount done 3.0
INFO [2026-09-25 07:58:50,649] [cassandra:b3e0f860-b8b6-11f1-a726-ebd0550bac6c] i.c.s.RepairRunner - Repairs for repair run #b3e0f860-b8b6-11f1-a726-ebd0550bac6c done