Skip to content

CyberAgentGameEntertainment/InstantReplay

Repository files navigation

Instant Replay for Unity

日本語

Instant Replay is a Unity library for saving recent gameplay footage on demand. It maintains a rolling buffer, so you can save important moments even after they happen. Recording is limited to a preconfigured duration, and frames older than that limit are discarded.

For Sharing Gameplay on Social Media

You can implement a feature that allows users to share their recent gameplay footage on social media.

For Recording Bug Reproduction Steps

When a bug occurs, you can export the recent gameplay leading up to the bug as a video, which can help reproduce it.

Table of Contents

Requirements

  • Unity 2022.3 or later

Note

The following information is based on the APIs and platform tools being used, and actual functionality may not have been verified.

Platform OS version aarch64 x86_64 Other requirements
iOS 10.0+ N/A
Android 8.0+
macOS 11.0+
Windows Windows 10+, Windows Server 2016+ -
Linux kernel 3.2+, glibc 2.17+ - ffmpeg in PATH
Web (any) (any) (any) Browser supports WebCodecs
  • For legacy mode, other platforms may work if ffmpeg is available in PATH.

Warning

Known Issue with WebGL: In WebGL, flickering may occur on the screen during recording. This is caused by ScreenshotFrameProvider, which is the default IFrameProvider implementation. If you encounter this issue, please use BuiltinCameraFrameProvider (for Built-in RP), RendererFeatureFrameProvider (for Universal RP), or another custom IFrameProvider implementation that provides input RenderTexture directly.

Encoder APIs in use

Platform APIs
iOS / macOS Video Toolbox (H.264), Audio Toolbox (AAC)
Android MediaCodec (H.264 / AAC)
Windows Media Foundation (H.264 / AAC)
Linux FFmpeg installed on the system (H.264 / AAC)
Web WebCodecs (avc1.640028 for video, mp4a.40.2 for audio)

Installation

Install Dependencies

You can install the dependencies using either of the following methods.

Method 1: Install via UnityNuGet and dependency package

Add UnityNuGet scoped registry and add the following git URL to the Package Manager:

https://github.com/CyberAgentGameEntertainment/InstantReplay.git?path=/Packages/jp.co.cyberagent.instant-replay.dependencies#release

Method 2: Install manually

Install individual packages using NuGetForUnity or UnityNuGet:

Install the Package

Add the following git URL to the Package Manager:

https://github.com/CyberAgentGameEntertainment/InstantReplay.git?path=Packages/jp.co.cyberagent.instant-replay#release

Quick Start

Import the "User Interfaces" sample from the Package Manager.

Image

Place InstantReplay Recorder.prefab in the scene. This prefab contains RecorderInterface and PersistentRecorder components, which automatically record the gameplay while enabled.

Image

Then, you can stop the recording and save the video by calling RecorderInterface.StopAndExport(). For example, you can trigger this method by clicking the button in the scene.

Image

The recorded video will play on the screen.

image

Detailed Usage

To record gameplay from code, use RealtimeInstantReplaySession as shown below.

using InstantReplay;

var ct = destroyCancellationToken;

// Start recording
using var session = RealtimeInstantReplaySession.CreateDefault();

// 〜 Gameplay 〜
await Task.Delay(10000, ct);

// Stop recording and transcode
var outputPath = await session.StopAndExportAsync();
File.Move(outputPath, Path.Combine(Application.persistentDataPath, Path.GetFileName(outputPath)));

Options

Recording uses memory in two places: buffers for compressed output, and buffers for raw frames and audio samples awaiting encoding.

MaxMemoryUsageBytesForCompressedFrames controls the recording duration. By default, recording holds up to 20 MiB of compressed data; when the total size of compressed frames and audio samples reaches this limit, older data is discarded. To enable longer recordings, increase this value or reduce the frame rate, resolution, or bitrate.

VideoInputQueueSize, AudioInputQueueSizeSeconds, and MaxNumberOfRawFrameBuffers (optional) control how many raw frames and audio samples are queued for encoding. These queues are needed because the encoder runs asynchronously, receiving the next frame while encoding the current one. Reducing these values decreases memory usage but may increase the likelihood of dropped frames.

// Default settings
var options = new RealtimeEncodingOptions
{
    VideoOptions = new VideoEncoderOptions
    {
        Width = 1280,
        Height = 720,
        FpsHint = 30,
        Bitrate = 2500000 // 2.5 Mbps
    },
    AudioOptions = new AudioEncoderOptions
    {
        SampleRate = 44100,
        Channels = 2,
        Bitrate = 128000 // 128 kbps
    },
    MaxNumberOfRawFrameBuffers = 2, // (Optional) Max number of buffers to store frames to be encoded. Each buffer size is VideoOptions.Width * VideoOptions.Height * 4 bytes.
    MaxMemoryUsageBytesForCompressedFrames = 20 * 1024 * 1024, // 20 MiB
    FixedFrameRate = 30.0, // null to use the actual rendering frame rate
    VideoInputQueueSize = 5, // Maximum number of raw frames to keep before encoding
    AudioInputQueueSizeSeconds = 1.0 // Max queued audio input duration to be buffered before encoding, in seconds
};

using var session = new RealtimeInstantReplaySession(options);

Pausing and Resuming

You can pause and resume the recording using RealtimeInstantReplaySession.Pause() and RealtimeInstantReplaySession.Resume().

Setting the Video Source

You can use a custom video source by implementing IFrameProvider.

Pass an IFrameProvider instance as frameProvider to the RealtimeInstantReplaySession constructor. You can also specify whether RealtimeInstantReplaySession automatically disposes frameProvider via the disposeFrameProvider parameter.

new RealtimeInstantReplaySession(options, frameProvider: new ScreenshotFrameProvider(), disposeFrameProvider: true);

Built-in IFrameProvider

  • ScreenshotFrameProvider: This is the default IFrameProvider implementation. It uses ScreenCapture.CaptureScreenshotIntoRenderTexture(), which allows it to capture the entire screen, including overlay canvases that are not rendered by a specific camera. However, it increases GPU memory usage due to the additional RenderTexture used for capturing.
  • BuiltinCameraFrameProvider: Captures the footage of a specific camera using OnRenderImage() in Built-in Render Pipeline.
  • RendererFeatureFrameProvider: Captures the footage of a specific camera using Renderer Feature in Universal Render Pipeline. You need to add InstantReplayFrameRendererFeature to the Renderer used by the camera.

Custom IFrameProvider Implementation

Create a class that inherits InstantReplay.IFrameProvider.

public interface IFrameProvider : IDisposable
{
    public delegate void ProvideFrame(Frame frame);

    event ProvideFrame OnFrameProvided;
}

new RealtimeInstantReplaySession(options, frameProvider: new CustomFrameProvider(), disposeFrameProvider: true);

Setting the Audio Source

By default, RealtimeInstantReplaySession captures the audio via OnAudioFilterRead. This automatically searches for and uses a specific AudioListener in the scene.

Warning

AudioSources with Bypass Listener Effects enabled will not be captured.

If there are multiple AudioListeners in the scene, create a UnityAudioSampleProvider with the one you want to use. Then pass it to the RealtimeInstantReplaySession constructor via the audioSampleProvider parameter.

new RealtimeInstantReplaySession(options, audioSampleProvider: new UnityAudioSampleProvider(audioListener), disposeAudioSampleProvider: true);

If you want to disable the audio, you can use NullAudioSampleProvider.Instance.

new RealtimeInstantReplaySession(options, audioSampleProvider: NullAudioSampleProvider.Instance);

Note

You don't need to dispose NullAudioSampleProvider.Instance because it is a shared singleton.

You can also use your own audio source by implementing IAudioSampleProvider.

public interface IAudioSampleProvider : IDisposable
{
    public delegate void ProvideAudioSamples(ReadOnlySpan<float> samples, int channels, int sampleRate,
        double timestamp);

    event ProvideAudioSamples OnProvideAudioSamples;
}

new RealtimeInstantReplaySession(options, audioSampleProvider: new CustomAudioSampleProvider(), disposeAudioSampleProvider: true);

CRI Support

InstantReplay provides CriAudioSampleProvider, an IAudioSampleProvider implementation that captures audio from CRIWARE.

  1. Install CRIWARE Unity Plug-in
  2. Add scripting define symbol INSTANTREPLAY_CRI in Player Settings
  3. Add InstantReplay.Cri assembly reference if necessary
  4. Use InstantReplay.Cri.CriAudioSampleProvider as audioSampleProvider in RealtimeInstantReplaySession constructor

Getting the Recording State

You can get the recording state with the RealtimeInstantReplaySession.State property.

Unbounded Recording

UnboundedRecordingSession writes encoded data directly to an MP4 file on disk without keeping it in memory, enabling unbounded recording limited only by available disk space. Apart from the required output file path in the constructor, UnboundedRecordingSession is used similarly to RealtimeInstantReplaySession.

Warning

If the app goes to the background during recording, the recording may stop and the recorded file may become corrupted. It is recommended to complete the recording before the app goes to the background.

using InstantReplay;

var ct = destroyCancellationToken;

// Start recording
using var session = new UnboundedRecordingSession("out.mp4", RealtimeEncodingOptions.Default);

// 〜 Gameplay 〜
await Task.Delay(10000, ct);

// Stop recording and export
await session.CompleteAsync();

Legacy Mode

By default, RealtimeInstantReplaySession encodes video and audio samples in real-time, but legacy InstantReplaySession saves JPEG-compressed video frames and raw audio samples to disk and transcodes them to a video file when StopAndTranscodeAsync is called. While this mode has higher disk I/O, it reduces the CPU load during recording.

using InstantReplay;

var ct = destroyCancellationToken;

// Start recording
using var session = new InstantReplaySession(numFrames: 900, fixedFrameRate: 30);

// 〜 Gameplay 〜
await Task.Delay(10000, ct);

// Stop recording and transcode
var outputPath = await session.StopAndTranscodeAsync(ct: ct);
File.Move(outputPath, Path.Combine(Application.persistentDataPath, Path.GetFileName(outputPath)));

Setting Recording Time and Frame Rate

You can specify numFrames and fixedFrameRate in the InstantReplaySession constructor.

new InstantReplaySession(numFrames: 900, fixedFrameRate: 30);

If you set fixedFrameRate to null, the actual frame rate will be used. When the number of frames exceeds numFrames, the oldest frames are discarded. The disk usage during recording increases in proportion to numFrames, so set it to an appropriate size.

Setting the Size

By default, recordings use the actual screen size. You can cap the output resolution with maxWidth and maxHeight in the InstantReplaySession constructor. Reducing the size lowers disk usage, write time, and memory usage during recording.

Video and Audio Sources

InstantReplaySession also supports custom video and audio sources in the same way as RealtimeInstantReplaySession.

Excluding from the Release Builds

If you are using InstantReplay as part of your bug reporting workflow, you should exclude script and plugin files in your release builds.

You can exclude all library scripts by adding EXCLUDE_INSTANTREPLAY to the Scripting Define Symbols in the Player Settings. To exclude your own code from the release builds, wrap it in #if !EXCLUDE_INSTANTREPLAY.

License

MIT

For the licenses of the dependencies used, please refer to THIRD-PARTY-NOTICES.md.

About

Record the latest seconds of gameplay and export it as a video at any time.

Resources

License

Stars

Watchers

Forks

Packages

 
 
 

Contributors