Skip to main content

VisionCamera

Classes​

Interfaces​

Type Aliases​

AutoFocusSystem​

Ƭ AutoFocusSystem: "contrast-detection" | "phase-detection" | "none"

Indicates a format's autofocus system.

  • "none": Indicates that autofocus is not available
  • "contrast-detection": Indicates that autofocus is achieved by contrast detection. Contrast detection performs a focus scan to find the optimal position
  • "phase-detection": Indicates that autofocus is achieved by phase detection. Phase detection has the ability to achieve focus in many cases without a focus scan. Phase detection autofocus is typically less visually intrusive than contrast detection autofocus

Defined in​

types/CameraDevice.ts:34

CameraPermissionRequestResult​

Ƭ CameraPermissionRequestResult: "granted" | "denied"

Defined in​

Camera.tsx:23

CameraPermissionStatus​

Ƭ CameraPermissionStatus: "granted" | "not-determined" | "denied" | "restricted"

Defined in​

Camera.tsx:22

CameraPosition​

Ƭ CameraPosition: "front" | "back" | "external"

Represents the camera device position.

  • "back": Indicates that the device is physically located on the back of the phone
  • "front": Indicates that the device is physically located on the front of the phone
  • "external": The camera device is an external camera, and has no fixed facing relative to the phone. (e.g. USB or Continuity Cameras)

Defined in​

types/CameraDevice.ts:10

CaptureError​

Ƭ CaptureError: "capture/recording-in-progress" | "capture/recording-canceled" | "capture/no-recording-in-progress" | "capture/file-io-error" | "capture/create-temp-file-error" | "capture/create-recorder-error" | "capture/insufficient-storage" | "capture/video-not-enabled" | "capture/photo-not-enabled" | "capture/frame-invalid" | "capture/no-data" | "capture/recorder-error" | "capture/focus-canceled" | "capture/focus-requires-preview" | "capture/timed-out" | "capture/snapshot-failed" | "capture/snapshot-failed-preview-not-enabled" | "capture/image-data-access-error" | "capture/encoder-error" | "capture/invalid-image-type" | "capture/failed-writing-metadata" | "capture/unknown"

Defined in​

CameraError.ts:37

CodeScannerError​

Ƭ CodeScannerError: "code-scanner/not-compatible-with-outputs" | "code-scanner/code-type-not-supported" | "code-scanner/cannot-load-model"

Defined in​

CameraError.ts:33

CodeType​

Ƭ CodeType: "code-128" | "code-39" | "code-93" | "codabar" | "ean-13" | "ean-8" | "itf" | "upc-e" | "upc-a" | "qr" | "pdf-417" | "aztec" | "data-matrix"

The type of the code to scan.

Defined in​

types/CodeScanner.ts:6

DeviceError​

Ƭ DeviceError: "device/configuration-error" | "device/no-device" | "device/invalid-device" | "device/microphone-unavailable" | "device/pixel-format-not-supported" | "device/low-light-boost-not-supported" | "device/focus-not-supported" | "device/camera-not-available-on-simulator" | "device/camera-already-in-use"

Defined in​

CameraError.ts:7

FormatError​

Ƭ FormatError: "format/invalid-fps" | "format/invalid-video-hdr" | "format/photo-hdr-and-video-hdr-not-suppoted-simultaneously" | "format/low-light-boost-not-supported-with-hdr" | "format/invalid-video-stabilization-mode" | "format/incompatible-pixel-format-with-hdr-setting" | "format/invalid-format" | "format/format-required"

Defined in​

CameraError.ts:17

Orientation​

Ƭ Orientation: "portrait" | "portrait-upside-down" | "landscape-left" | "landscape-right"

Represents Orientation. Depending on the context, this might be a sensor orientation (relative to the phone's orentation), or view orientation.

  • portrait: 0° (home-button at the bottom)
  • landscape-left: 90° (home-button on the left)
  • portrait-upside-down: 180° (home-button at the top)
  • landscape-right: 270° (home-button on the right)

Defined in​

types/Orientation.ts:10

ParameterError​

Ƭ ParameterError: "parameter/invalid-parameter" | "parameter/unsupported-output" | "parameter/unsupported-input" | "parameter/invalid-combination"

Defined in​

CameraError.ts:2

ParameterType​

Ƭ ParameterType: BasicParameterType | BasicParameterType[] | Record<string, BasicParameterType | undefined>

Defined in​

frame-processors/initFrameProcessorPlugin.ts:5

PermissionError​

Ƭ PermissionError: "permission/microphone-permission-denied" | "permission/camera-permission-denied"

Defined in​

CameraError.ts:1

PhysicalCameraDeviceType​

Ƭ PhysicalCameraDeviceType: "ultra-wide-angle-camera" | "wide-angle-camera" | "telephoto-camera"

Indentifiers for a physical camera (one that actually exists on the back/front of the device)

  • "ultra-wide-angle-camera": A built-in camera with a shorter focal length than that of a wide-angle camera. (FOV of 94° or higher)
  • "wide-angle-camera": A built-in wide-angle camera. (FOV between 60° and 94°)
  • "telephoto-camera": A built-in camera device with a longer focal length than a wide-angle camera. (FOV of 60° or lower)

Some Camera devices consist of multiple physical devices. They can be interpreted as logical devices, for example:

  • "ultra-wide-angle-camera" + "wide-angle-camera" = dual wide-angle camera.
  • "wide-angle-camera" + "telephoto-camera" = dual camera.
  • "ultra-wide-angle-camera" + "wide-angle-camera" + "telephoto-camera" = triple camera.

Defined in​

types/CameraDevice.ts:25

PixelFormat​

Ƭ PixelFormat: "yuv" | "rgb" | "unknown"

Represents the pixel format of a Frame.

If you intend to read Pixels from this Frame or use an ML model for processing, make sure that you are using the expected PixelFormat, otherwise the plugin might not be able to properly understand the Frame's content.

Most ML models operate in either yuv (recommended) or rgb.

  • yuv: Frame is in YUV pixel-format (Y'CbCr 4:2:0 or NV21, 8-bit)
  • rgb: Frame is in RGB pixel-format (RGBA or BGRA, 8-bit)
  • unknown: Frame has unknown/unsupported pixel-format.

Defined in​

types/PixelFormat.ts:13

SessionError​

Ƭ SessionError: "session/camera-not-ready" | "session/audio-in-use-by-other-app" | "session/no-outputs" | "session/audio-session-failed-to-activate" | "session/hardware-cost-too-high" | "session/invalid-output-configuration"

Defined in​

CameraError.ts:26

SnapshotFile​

Ƭ SnapshotFile: Pick<PhotoFile, "path" | "width" | "height" | "orientation" | "isMirrored">

Defined in​

types/Snapshot.ts:11

SystemError​

Ƭ SystemError: "system/camera-module-not-found" | "system/camera-is-restricted" | "system/location-not-enabled" | "system/no-camera-manager" | "system/frame-processors-unavailable" | "system/recording-while-frame-processing-unavailable" | "system/view-not-found" | "system/max-cameras-in-use" | "system/do-not-disturb-bug"

Defined in​

CameraError.ts:60

UnknownError​

Ƭ UnknownError: "unknown/unknown"

Defined in​

CameraError.ts:70

VideoStabilizationMode​

Ƭ VideoStabilizationMode: "off" | "standard" | "cinematic" | "cinematic-extended" | "auto"

Indicates a format's supported video stabilization mode. Enabling video stabilization may introduce additional latency into the video capture pipeline.

  • "off": No video stabilization. Indicates that video should not be stabilized
  • "standard": Standard software-based video stabilization. Standard video stabilization reduces the field of view by about 10%.
  • "cinematic": Advanced software-based video stabilization. This applies more aggressive cropping or transformations than standard.
  • "cinematic-extended": Extended software- and hardware-based stabilization that aggressively crops and transforms the video to apply a smooth cinematic stabilization.
  • "auto": Indicates that the most appropriate video stabilization mode for the device and format should be chosen automatically

Defined in​

types/CameraDevice.ts:45

Variables​

Templates​

• Const Templates: Record<PredefinedTemplates, FormatFilter[]>

Predefined templates for use in useCameraFormat/getCameraFormat.

Example

const format = useCameraFormat(device, Templates.Snapchat)

Defined in​

devices/Templates.ts:26

VisionCameraProxy​

• Const VisionCameraProxy: TVisionCameraProxy

The JSI Proxy for the Frame Processors Runtime.

This will be replaced with a CxxTurboModule in the future.

Defined in​

frame-processors/VisionCameraProxy.ts:38

Functions​

createFrameProcessor​

â–¸ createFrameProcessor(frameProcessor): ReadonlyFrameProcessor

Create a new Frame Processor function which you can pass to the <Camera>. (See "Frame Processors")

Make sure to add the 'worklet' directive to the top of the Frame Processor function, otherwise it will not get compiled into a worklet.

Also make sure to memoize the returned object, so that the Camera doesn't reset the Frame Processor Context each time.

Parameters​

NameType
frameProcessor(frame: Frame) => void

Returns​

ReadonlyFrameProcessor

Worklet

Defined in​

hooks/useFrameProcessor.ts:16

createSkiaFrameProcessor​

â–¸ createSkiaFrameProcessor(frameProcessor, surfaceHolder, offscreenTextures): DrawableFrameProcessor

Create a new Frame Processor function which you can pass to the <Camera>. (See "Frame Processors")

Make sure to add the 'worklet' directive to the top of the Frame Processor function, otherwise it will not get compiled into a worklet.

Also make sure to memoize the returned object, so that the Camera doesn't reset the Frame Processor Context each time.

Parameters​

NameType
frameProcessor(frame: DrawableFrame) => void
surfaceHolderISharedValue<SurfaceCache>
offscreenTexturesISharedValue<SkImage[]>

Returns​

DrawableFrameProcessor

Worklet

Example

const surfaceHolder = Worklets.createSharedValue<SurfaceCache>({})
const offscreenTextures = Worklets.createSharedValue<SkImage[]>([])
const frameProcessor = createSkiaFrameProcessor((frame) => {
  'worklet'
  const faces = scanFaces(frame)

  frame.render()
  for (const face of faces) {
    const rect = Skia.XYWHRect(face.x, face.y, face.width, face.height)
    frame.drawRect(rect)
  }
}, surfaceHolder, offscreenTextures)

Defined in​

skia/useSkiaFrameProcessor.ts:105

getCameraDevice​

â–¸ getCameraDevice(devices, position, filter?): CameraDevice | undefined

Get the best matching Camera device that best satisfies your requirements using a sorting filter, or undefined if devices does not contain any devices.

Parameters​

NameTypeDescription
devicesCameraDevice[]-
positionCameraPositionThe position of the Camera device relative to the phone.
filterDeviceFilterThe filter you want to use. The Camera device that matches your filter the closest will be returned

Returns​

CameraDevice | undefined

The Camera device that matches your filter the closest, or undefined if no such Camera Device exists on the given position.

Example

const devices = Camera.getAvailableCameraDevices()
const device = getCameraDevice(devices, 'back', {
   physicalDevices: ['wide-angle-camera']
})

Defined in​

devices/getCameraDevice.ts:38

getCameraFormat​

â–¸ getCameraFormat(device, filters): CameraDeviceFormat

Get the best matching Camera format for the given device that satisfies your requirements using a sorting filter. By default, formats are sorted by highest to lowest resolution.

The filters are ranked by priority, from highest to lowest. This means the first item you pass will have a higher priority than the second, and so on.

Parameters​

NameTypeDescription
deviceCameraDeviceThe Camera Device you're currently using
filtersFormatFilter[]The filters you want to use. The format that matches your filter the closest will be returned

Returns​

CameraDeviceFormat

The format that matches your filter the closest.

Example

const format = getCameraFormat(device, [
  { videoResolution: { width: 3048, height: 2160 } },
  { fps: 60 }
])

Defined in​

devices/getCameraFormat.ts:119

initFrameProcessorPlugin​

â–¸ initFrameProcessorPlugin(name, options?): FrameProcessorPlugin | undefined

Creates a new instance of a native Frame Processor Plugin. The Plugin has to be registered on the native side, otherwise this returns undefined.

Parameters​

NameTypeDescription
namestringThe name of the Frame Processor Plugin. This has to be the same name as on the native side.
optionsRecord<string, ParameterType>(optional) Options, as a native dictionary, passed to the constructor/init-function of the native plugin.

Returns​

FrameProcessorPlugin | undefined

Example

const plugin = VisionCameraProxy.initFrameProcessorPlugin('scanFaces', { model: 'fast' })
if (plugin == null) throw new Error("Failed to load scanFaces plugin!")

Defined in​

frame-processors/initFrameProcessorPlugin.ts:32

isErrorWithCause​

â–¸ isErrorWithCause(error): error is ErrorWithCause

Checks if the given error is of type ErrorWithCause

Parameters​

NameTypeDescription
errorunknownAny unknown object to validate

Returns​

error is ErrorWithCause

true if the given error is of type ErrorWithCause

Defined in​

CameraError.ts:188

runAsync​

â–¸ runAsync(frame, func): void

Runs the given func asynchronously on a separate thread, allowing the Frame Processor to continue executing without dropping a Frame.

Only one runAsync call will execute at the same time, so runAsync is not parallel, but asynchronous.

For example, if your Camera is running at 60 FPS (16ms per frame), and a heavy ML face detection Frame Processor Plugin takes 500ms to execute, you have two options:

  • Run the plugin normally (synchronously in useFrameProcessor) but drop a lot of Frames, as we can only run at 2 FPS (500ms per frame)
  • Call the plugin inside runAsync to allow the Camera to still run at 60 FPS, but offload the heavy ML face detection plugin to the asynchronous context, where it will run at 2 FPS.

Parameters​

NameTypeDescription
frameFrameThe current Frame of the Frame Processor.
func() => voidThe function to execute.

Returns​

void

Note

runAsync cannot be used to draw to a Frame in a Skia Frame Processor.

Worklet

Example

const frameProcessor = useFrameProcessor((frame) => {
  'worklet'
  console.log('New Frame arrived!')

  runAsync(frame, () => {
    'worklet'
    const faces = detectFaces(frame)
    const face = [faces0]
    console.log(`Detected a new face: ${face}`)
  })
})

Defined in​

frame-processors/runAsync.ts:83

runAtTargetFps​

â–¸ runAtTargetFps<T>(fps, func): T | undefined

Runs the given func at the given target fps rate.

runAtTargetFps still executes the given func synchronously, so this is only useful for throttling calls to a plugin or logger.

For example, if you want to scan faces only once per second to avoid excessive CPU usage, use runAtTargetFps(1, ...).

Type parameters​

Name
T

Parameters​

NameTypeDescription
fpsnumberThe target FPS rate at which the given function should be executed
func() => TThe function to execute.

Returns​

T | undefined

The result of the function if it was executed, or undefined otherwise.

Worklet

Example

const frameProcessor = useFrameProcessor((frame) => {
  'worklet'
  console.log('New Frame')
  runAtTargetFps(5, () => {
    'worklet'
    const faces = detectFaces(frame)
    console.log(`Detected a new face: ${faces[0]}`)
  })
})

Defined in​

frame-processors/runAtTargetFps.ts:43

tryParseNativeCameraError​

â–¸ tryParseNativeCameraError<T>(nativeError): CameraCaptureError | CameraRuntimeError | T

Tries to parse an error coming from native to a typed JS camera error.

Type parameters​

Name
T

Parameters​

NameTypeDescription
nativeErrorTThe native error instance. This is a JSON in the legacy native module architecture.

Returns​

CameraCaptureError | CameraRuntimeError | T

A CameraRuntimeError or CameraCaptureError, or the nativeError itself if it's not parsable

Method

Defined in​

CameraError.ts:214

useCameraDevice​

â–¸ useCameraDevice(position, filter?): CameraDevice | undefined

Get the best matching Camera device that best satisfies your requirements using a sorting filter.

Parameters​

NameTypeDescription
positionCameraPositionThe position of the Camera device relative to the phone.
filter?DeviceFilterThe filter you want to use. The Camera device that matches your filter the closest will be returned

Returns​

CameraDevice | undefined

The Camera device that matches your filter the closest, or undefined if no such Camera Device exists on the given position.

Example

const device = useCameraDevice('back', {
   physicalDevices: ['wide-angle-camera']
})

Defined in​

hooks/useCameraDevice.ts:19

useCameraDevices​

â–¸ useCameraDevices(): CameraDevice[]

Get all available Camera Devices this phone has.

Camera Devices attached to this phone (back or front) are always available, while external devices might be plugged in or out at any point, so the result of this function might update over time.

Returns​

CameraDevice[]

Defined in​

hooks/useCameraDevices.ts:12

useCameraFormat​

â–¸ useCameraFormat(device, filters): CameraDeviceFormat | undefined

Get the best matching Camera format for the given device that satisfies your requirements using a sorting filter. By default, formats are sorted by highest to lowest resolution.

The filters are ranked by priority, from highest to lowest. This means the first item you pass will have a higher priority than the second, and so on.

Parameters​

NameTypeDescription
deviceundefined | CameraDeviceThe Camera Device you're currently using
filtersFormatFilter[]The filters you want to use. The format that matches your filter the closest will be returned

Returns​

CameraDeviceFormat | undefined

The format that matches your filter the closest.

Example

const device = useCameraDevice(...)
const format = useCameraFormat(device, [
  { videoResolution: { width: 3048, height: 2160 } },
  { fps: 60 }
])

Defined in​

hooks/useCameraFormat.ts:24

useCameraPermission​

â–¸ useCameraPermission(): PermissionState

Returns whether the user has granted permission to use the Camera, or not.

If the user doesn't grant Camera Permission, you cannot use the <Camera>.

Returns​

PermissionState

Example

const { hasPermission, requestPermission } = useCameraPermission()

if (!hasPermission) {
  return <PermissionScreen onPress={requestPermission} />
} else {
  return <Camera ... />
}

Defined in​

hooks/useCameraPermission.ts:62

useCodeScanner​

â–¸ useCodeScanner(codeScanner): CodeScanner

Parameters​

NameType
codeScannerCodeScanner

Returns​

CodeScanner

Defined in​

hooks/useCodeScanner.ts:4

useFrameProcessor​

â–¸ useFrameProcessor(frameProcessor, dependencies): ReadonlyFrameProcessor

Returns a memoized Frame Processor function wich you can pass to the <Camera>. (See "Frame Processors")

Make sure to add the 'worklet' directive to the top of the Frame Processor function, otherwise it will not get compiled into a worklet.

Parameters​

NameTypeDescription
frameProcessor(frame: Frame) => voidThe Frame Processor
dependenciesDependencyListThe React dependencies which will be copied into the VisionCamera JS-Runtime.

Returns​

ReadonlyFrameProcessor

The memoized Frame Processor.

Worklet

Example

const frameProcessor = useFrameProcessor((frame) => {
  'worklet'
  const faces = scanFaces(frame)
  console.log(`Faces: ${faces}`)
}, [])

Defined in​

hooks/useFrameProcessor.ts:42

useLocationPermission​

â–¸ useLocationPermission(): PermissionState

Returns whether the user has granted permission to use the Location, or not.

If the user doesn't grant Location Permission, you can use the <Camera> but you cannot capture photos or videos with GPS EXIF tags (the location={..} prop).

Returns​

PermissionState

Example

const { hasPermission, requestPermission } = useLocationPermission()
const canCaptureLocation = hasPermission

return <Camera photo={true} location={canCaptureLocation} />

Defined in​

hooks/useCameraPermission.ts:98

useMicrophonePermission​

â–¸ useMicrophonePermission(): PermissionState

Returns whether the user has granted permission to use the Microphone, or not.

If the user doesn't grant Audio Permission, you can use the <Camera> but you cannot record videos with audio (the audio={..} prop).

Returns​

PermissionState

Example

const { hasPermission, requestPermission } = useMicrophonePermission()
const canRecordAudio = hasPermission

return <Camera video={true} audio={canRecordAudio} />

Defined in​

hooks/useCameraPermission.ts:80

useSkiaFrameProcessor​

â–¸ useSkiaFrameProcessor(frameProcessor, dependencies): DrawableFrameProcessor

Returns a memoized Skia Frame Processor function wich you can pass to the <Camera>.

The Skia Frame Processor alows you to draw ontop of the Frame, and will manage it's internal offscreen Skia Canvas and onscreen Skia preview view.

(See "Frame Processors")

Make sure to add the 'worklet' directive to the top of the Frame Processor function, otherwise it will not get compiled into a worklet.

Parameters​

NameTypeDescription
frameProcessor(frame: DrawableFrame) => voidThe Frame Processor
dependenciesDependencyListThe React dependencies which will be copied into the VisionCamera JS-Runtime.

Returns​

DrawableFrameProcessor

The memoized Skia Frame Processor.

Worklet

Example

const frameProcessor = useSkiaFrameProcessor((frame) => {
  'worklet'
  const faces = scanFaces(frame)

  frame.render()
  for (const face of faces) {
    const rect = Skia.XYWHRect(face.x, face.y, face.width, face.height)
    frame.drawRect(rect)
  }
}, [])

Defined in​

skia/useSkiaFrameProcessor.ts:273