The Realtime SDK does not work with React Native #133
Description
React Native compatibility.
I'm using the Agents SDK in a React Native (+ Expo) application. However the realtime SDK does not work. I was able to build a custom client but quickly realized I wasn't getting traces for instance..
I'm not sure what would be a good solution, it feels like the changes to make this compatible would be extensive..
FYI here is what my robot friends told me it would entail:
React Native Support for @openai/agents-realtime
Overview
This issue proposes adding React Native support to the @openai/agents-realtime package. Currently, the SDK only supports Node.js and browser environments, but there's significant demand for React Native support to build mobile voice agents.
Motivation
- Growing demand for voice AI agents in mobile applications
- React Native is a popular framework for cross-platform mobile development
- Current SDK uses Node.js-specific dependencies that don't work in React Native
- Community members are building custom implementations (evidence of need)
Proposed Changes
1. WebSocket Implementation
React Native has its own WebSocket implementation that differs from Node.js ws package.
// Current implementation
import WebSocket from 'ws'; // Node.js only
// Proposed change
const WebSocket = Platform.select({
web: () => window.WebSocket,
default: () => require('react-native').WebSocket
})();2. WebRTC Integration
The most significant change - React Native requires the react-native-webrtc package.
// Current: Browser-native WebRTC
const pc = new RTCPeerConnection(config);
// Proposed: React Native WebRTC
import {
RTCPeerConnection,
RTCSessionDescription,
RTCIceCandidate,
mediaDevices,
MediaStream,
MediaStreamTrack,
registerGlobals
} from 'react-native-webrtc';
// Must call this once at app startup
registerGlobals();3. Platform Detection Utility
Add platform detection throughout the codebase:
export const isReactNative = () => {
return typeof navigator !== 'undefined' &&
navigator.product === 'ReactNative';
};
// Usage
if (isReactNative()) {
// React Native specific code
} else if (typeof window !== 'undefined') {
// Browser code
} else {
// Node.js code
}4. Module Import Strategy
Replace Node.js built-in modules with cross-platform alternatives:
- Remove imports of
http,https,crypto,events - Use conditional imports based on platform
- Provide polyfills where necessary
5. Event Emitter Compatibility
// Current
import { EventEmitter } from 'events';
// Proposed
import { EventEmitter } from 'eventemitter3'; // Cross-platform6. TypeScript Type Compatibility
// Current
private timer: NodeJS.Timeout;
// Proposed
private timer: number | NodeJS.Timeout; // Support both environments7. Audio Session Management
React Native requires explicit audio session configuration:
import { Audio } from 'expo-av';
// Configure before starting WebRTC
await Audio.setAudioModeAsync({
allowsRecordingIOS: true,
playsInSilentModeIOS: true,
staysActiveInBackground: false,
shouldDuckAndroid: true,
playThroughEarpieceAndroid: false,
});8. Permission Handling
import { PermissionsAndroid, Platform } from 'react-native';
async function requestMicrophonePermission() {
if (Platform.OS === 'android') {
const granted = await PermissionsAndroid.request(
PermissionsAndroid.PERMISSIONS.RECORD_AUDIO
);
return granted === PermissionsAndroid.RESULTS.GRANTED;
}
return true; // iOS permissions via Info.plist
}9. Package Configuration
Update package.json to support React Native:
{
"main": "dist/index.js",
"react-native": "dist/index.native.js",
"browser": "dist/index.browser.js",
"exports": {
".": {
"react-native": "./dist/index.native.js",
"browser": "./dist/index.browser.js",
"node": "./dist/index.js"
}
},
"peerDependencies": {
"react-native-webrtc": "^118.0.0"
}
}Implementation Plan
Phase 1: Core Compatibility
- Add platform detection utilities
- Create React Native-specific transport layer
- Implement WebRTC integration with react-native-webrtc
- Add cross-platform event emitter
Phase 2: Audio & Permissions
- Add audio session management
- Implement permission request flows
- Handle platform-specific audio formats
Phase 3: Build & Testing
- Set up React Native build pipeline
- Create example React Native app
- Add React Native test suite
- Test on iOS and Android devices
Phase 4: Documentation
- Add React Native installation guide
- Document platform-specific setup
- Create troubleshooting guide
- Add API documentation for RN-specific features
Backwards Compatibility
All changes will be backwards compatible:
- Existing Node.js/browser functionality unchanged
- Platform-specific code only loaded when needed
- Same API surface across all platforms
- No breaking changes to existing users
Alternative Approaches
-
Separate Package: Create
@openai/agents-realtime-react-native- Pros: Complete isolation, no risk to existing package
- Cons: Code duplication, maintenance overhead
-
Monorepo Approach: Split into multiple packages
- Pros: Clean separation, tree-shaking benefits
- Cons: More complex setup, potential breaking change
-
Platform Detection (Recommended)
- Pros: Single package, best DX, automatic platform selection
- Cons: Slightly larger package size
Testing Requirements
- Unit tests for all platform-specific code
- Integration tests on real devices
- iOS 13+ and Android 10+ support
- Example app demonstrating all features
- CI/CD pipeline for React Native builds
Documentation Requirements
- Installation guide for React Native
- Platform-specific setup (iOS Info.plist, Android manifest)
- Troubleshooting common issues
- Migration guide for custom implementations
- Example code and best practices
Community Impact
This change would benefit:
- React Native developers building AI-powered mobile apps
- Companies wanting to add voice agents to mobile apps
- The broader React Native + AI ecosystem