feat: 完成前端例句口說練習功能和最終修復
🖥️ 前端例句口說練習完整實現: - AudioRecorder 共用組件 (Web Audio API 高品質錄音) - SentenceSpeakingQuiz 完整組件 (錄音/API/評分結果顯示) - speechAssessmentService.ts API 客戶端服務 - 完美整合到複習系統 (第3種 quiz type) 🔧 系統修復和優化: - 擴展 useReviewSession.ts 支援 sentence-speaking - 更新 reviewSimpleData.ts 類型定義 - 修復 review/page.tsx 條件渲染邏輯 - 優化 SentenceSpeakingQuiz 圖片顯示佈局 📋 技術規格文檔更新: - 更新開發進度和第一階段完成狀態 - 記錄所有實現的技術組件和驗證結果 🎨 用戶體驗優化: - 響應式圖片顯示設計 (max-w-lg, 300px 高度限制) - 智能無圖提示和有圖引導 - 完整的錄音狀態視覺反饋 - CEFR 等級顯示修復 現在 DramaLing 具備完整的 AI 驅動例句口說練習功能！ 包含圖片顯示、專業錄音、多維度 AI 評分、智能反饋 🎤✨ 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
9bebe78740
commit
e3bc290b56
|
|
@ -3,6 +3,7 @@
|
|||
import { Navigation } from '@/components/shared/Navigation'
|
||||
import { FlipMemory } from '@/components/review/quiz/FlipMemory'
|
||||
import { VocabChoiceQuiz } from '@/components/review/quiz/VocabChoiceQuiz'
|
||||
import { SentenceSpeakingQuiz } from '@/components/review/quiz/SentenceSpeakingQuiz'
|
||||
import { QuizProgress } from '@/components/review/ui/QuizProgress'
|
||||
import { QuizResult } from '@/components/review/quiz/QuizResult'
|
||||
import { useReviewSession } from '@/hooks/review/useReviewSession'
|
||||
|
|
@ -347,6 +348,14 @@ export default function ReviewPage() {
|
|||
onSkip={handleSkip}
|
||||
/>
|
||||
)}
|
||||
|
||||
{currentQuizItem.quizType === 'sentence-speaking' && (
|
||||
<SentenceSpeakingQuiz
|
||||
card={currentCard}
|
||||
onAnswer={handleAnswer}
|
||||
onSkip={handleSkip}
|
||||
/>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -0,0 +1,294 @@
|
|||
'use client'
|
||||
|
||||
import { useState, useCallback } from 'react'
|
||||
import { CardState } from '@/lib/data/reviewSimpleData'
|
||||
import { QuizHeader } from '../ui/QuizHeader'
|
||||
import { AudioRecorder } from '@/components/shared/AudioRecorder'
|
||||
import { speechAssessmentService, PronunciationResult } from '@/lib/services/speechAssessment'
|
||||
import { getFlashcardImageUrl } from '@/lib/utils/flashcardUtils'
|
||||
|
||||
interface SentenceSpeakingQuizProps {
|
||||
card: CardState
|
||||
onAnswer: (confidence: number) => void
|
||||
onSkip: () => void
|
||||
}
|
||||
|
||||
export function SentenceSpeakingQuiz({ card, onAnswer, onSkip }: SentenceSpeakingQuizProps) {
|
||||
const [assessmentResult, setAssessmentResult] = useState<PronunciationResult | null>(null)
|
||||
const [isEvaluating, setIsEvaluating] = useState(false)
|
||||
const [hasAnswered, setHasAnswered] = useState(false)
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
|
||||
// 獲取例句圖片 - 多重來源檢查
|
||||
const exampleImageUrl = getFlashcardImageUrl(card) ||
|
||||
(card as any).PrimaryImageUrl ||
|
||||
card.primaryImageUrl ||
|
||||
null
|
||||
|
||||
// 除錯資訊 - 檢查圖片資料
|
||||
console.log('🔍 SentenceSpeaking 圖片除錯:', {
|
||||
cardId: card.id,
|
||||
word: card.word,
|
||||
hasExampleImage: card.hasExampleImage,
|
||||
primaryImageUrl: card.primaryImageUrl,
|
||||
PrimaryImageUrl: (card as any).PrimaryImageUrl,
|
||||
exampleImages: (card as any).exampleImages,
|
||||
FlashcardExampleImages: (card as any).FlashcardExampleImages,
|
||||
originalCard: card,
|
||||
computedImageUrl: exampleImageUrl
|
||||
})
|
||||
|
||||
// 處理錄音完成
|
||||
const handleRecordingComplete = useCallback(async (audioBlob: Blob) => {
|
||||
if (hasAnswered) return
|
||||
|
||||
setIsEvaluating(true)
|
||||
setError(null)
|
||||
|
||||
try {
|
||||
console.log('🎤 開始發音評估...', {
|
||||
flashcardId: card.id,
|
||||
referenceText: card.example,
|
||||
audioSize: `${(audioBlob.size / 1024).toFixed(1)} KB`
|
||||
})
|
||||
|
||||
const response = await speechAssessmentService.evaluatePronunciation(
|
||||
audioBlob,
|
||||
card.example,
|
||||
card.id,
|
||||
'en-US'
|
||||
)
|
||||
|
||||
if (response.success && response.data) {
|
||||
setAssessmentResult(response.data)
|
||||
setHasAnswered(true)
|
||||
|
||||
// 稍後自動提交結果（給用戶時間查看評分）
|
||||
setTimeout(() => {
|
||||
onAnswer(response.data!.confidenceLevel)
|
||||
}, 2000)
|
||||
|
||||
} else {
|
||||
throw new Error(response.error || '發音評估失敗')
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
const errorMessage = error instanceof Error ? error.message : '評估過程發生錯誤'
|
||||
setError(errorMessage)
|
||||
console.error('發音評估錯誤:', error)
|
||||
|
||||
} finally {
|
||||
setIsEvaluating(false)
|
||||
}
|
||||
}, [hasAnswered, card.id, card.example, onAnswer])
|
||||
|
||||
// 處理跳過
|
||||
const handleSkip = useCallback(() => {
|
||||
onSkip()
|
||||
}, [onSkip])
|
||||
|
||||
// 手動提交結果（如果自動提交被取消）
|
||||
const handleSubmitResult = useCallback(() => {
|
||||
if (assessmentResult && hasAnswered) {
|
||||
onAnswer(assessmentResult.confidenceLevel)
|
||||
}
|
||||
}, [assessmentResult, hasAnswered, onAnswer])
|
||||
|
||||
// 重新錄音
|
||||
const handleRetry = useCallback(() => {
|
||||
setAssessmentResult(null)
|
||||
setHasAnswered(false)
|
||||
setError(null)
|
||||
}, [])
|
||||
|
||||
return (
|
||||
<div className="bg-white rounded-xl shadow-lg p-6">
|
||||
<QuizHeader
|
||||
title="例句口說練習"
|
||||
cefr={card.cefr || (card as any).CEFR || 'A1'}
|
||||
/>
|
||||
|
||||
<div className="space-y-6">
|
||||
{/* 說明文字 */}
|
||||
<div className="text-center bg-blue-50 border border-blue-200 rounded-lg p-4">
|
||||
<p className="text-blue-800 font-medium">
|
||||
📸 看圖說出完整例句，AI 將評估你的發音表現
|
||||
</p>
|
||||
</div>
|
||||
|
||||
{/* 例句圖片 - 更顯著的顯示 */}
|
||||
{exampleImageUrl ? (
|
||||
<div className="text-center">
|
||||
<div className="bg-gray-50 border border-gray-200 rounded-xl p-6 mb-4">
|
||||
<img
|
||||
src={exampleImageUrl}
|
||||
alt={`${card.word} example illustration`}
|
||||
className="w-full max-w-lg mx-auto rounded-lg shadow-sm"
|
||||
style={{ maxHeight: '300px', objectFit: 'contain' }}
|
||||
/>
|
||||
</div>
|
||||
<p className="text-gray-500 text-sm mb-6">
|
||||
💡 根據上圖理解情境，然後大聲說出例句
|
||||
</p>
|
||||
</div>
|
||||
) : (
|
||||
<div className="text-center bg-yellow-50 border border-yellow-200 rounded-lg p-4 mb-6">
|
||||
<p className="text-yellow-800 text-sm">
|
||||
⚠️ 此詞卡暫無例句圖片，請根據文字內容進行練習
|
||||
</p>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* 目標例句 */}
|
||||
<div className="bg-gradient-to-r from-green-50 to-blue-50 border border-green-200 rounded-lg p-6">
|
||||
<div className="text-center mb-4">
|
||||
<h4 className="text-lg font-semibold text-gray-900 mb-2">目標例句</h4>
|
||||
<p className="text-2xl text-gray-900 font-medium leading-relaxed mb-3">
|
||||
{card.example}
|
||||
</p>
|
||||
<p className="text-gray-600 text-base">
|
||||
{card.exampleTranslation}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* 錄音區域 */}
|
||||
{!hasAnswered && !isEvaluating && (
|
||||
<div>
|
||||
<AudioRecorder
|
||||
onRecordingComplete={handleRecordingComplete}
|
||||
onError={setError}
|
||||
maxDuration={30}
|
||||
className="mb-4"
|
||||
/>
|
||||
<p className="text-gray-500 text-sm text-center mb-4">
|
||||
💡 建議清晰地讀出完整句子，包含正確的語調和停頓
|
||||
</p>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* 評估中狀態 */}
|
||||
{isEvaluating && (
|
||||
<div className="bg-blue-50 border border-blue-200 rounded-lg p-6 text-center">
|
||||
<div className="animate-spin rounded-full h-8 w-8 border-b-2 border-blue-600 mx-auto mb-3"></div>
|
||||
<h4 className="font-semibold text-blue-900 mb-2">AI 正在評估發音...</h4>
|
||||
<p className="text-blue-700 text-sm">
|
||||
正在分析您的發音準確度、流暢度和語調 (約需 2-3 秒)
|
||||
</p>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* 錯誤顯示 */}
|
||||
{error && (
|
||||
<div className="bg-red-50 border border-red-200 rounded-lg p-4">
|
||||
<h4 className="font-semibold text-red-900 mb-2">❌ 評估失敗</h4>
|
||||
<p className="text-red-700 text-sm mb-3">{error}</p>
|
||||
<button
|
||||
onClick={handleRetry}
|
||||
className="px-4 py-2 bg-red-600 text-white rounded-lg hover:bg-red-700 transition-colors text-sm"
|
||||
>
|
||||
重新錄音
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* 評估結果顯示 */}
|
||||
{assessmentResult && hasAnswered && (
|
||||
<div className="bg-green-50 border border-green-200 rounded-lg p-6">
|
||||
<h4 className="font-semibold text-green-900 mb-4">🎉 發音評估結果</h4>
|
||||
|
||||
{/* 總分顯示 */}
|
||||
<div className="flex items-center justify-center gap-4 mb-4">
|
||||
<div className="text-center">
|
||||
<div className="text-4xl font-bold text-green-600 mb-1">
|
||||
{Math.round(assessmentResult.scores.overall)}
|
||||
</div>
|
||||
<div className="text-green-700 text-sm font-medium">總分</div>
|
||||
</div>
|
||||
<div className="text-green-600 text-lg">/ 100</div>
|
||||
</div>
|
||||
|
||||
{/* 詳細評分 */}
|
||||
<div className="grid grid-cols-2 gap-3 mb-4">
|
||||
<div className="bg-white p-3 rounded border">
|
||||
<div className="text-xs text-gray-600 mb-1">準確度</div>
|
||||
<div className="font-semibold text-lg text-gray-900">
|
||||
{Math.round(assessmentResult.scores.accuracy)}
|
||||
</div>
|
||||
</div>
|
||||
<div className="bg-white p-3 rounded border">
|
||||
<div className="text-xs text-gray-600 mb-1">流暢度</div>
|
||||
<div className="font-semibold text-lg text-gray-900">
|
||||
{Math.round(assessmentResult.scores.fluency)}
|
||||
</div>
|
||||
</div>
|
||||
<div className="bg-white p-3 rounded border">
|
||||
<div className="text-xs text-gray-600 mb-1">完整度</div>
|
||||
<div className="font-semibold text-lg text-gray-900">
|
||||
{Math.round(assessmentResult.scores.completeness)}
|
||||
</div>
|
||||
</div>
|
||||
<div className="bg-white p-3 rounded border">
|
||||
<div className="text-xs text-gray-600 mb-1">語調</div>
|
||||
<div className="font-semibold text-lg text-gray-900">
|
||||
{Math.round(assessmentResult.scores.prosody)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* 語音識別結果 */}
|
||||
{assessmentResult.transcribedText && (
|
||||
<div className="bg-white rounded border p-3 mb-4">
|
||||
<div className="text-xs text-gray-600 mb-2">AI 識別的內容</div>
|
||||
<div className="font-mono text-sm text-gray-900">
|
||||
{assessmentResult.transcribedText}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* 改善建議 */}
|
||||
{assessmentResult.feedback && assessmentResult.feedback.length > 0 && (
|
||||
<div className="space-y-2 mb-4">
|
||||
<div className="text-sm font-medium text-green-800">改善建議：</div>
|
||||
{assessmentResult.feedback.map((feedback, index) => (
|
||||
<div key={index} className="text-sm text-green-700 flex items-start gap-2">
|
||||
<span className="text-green-600 mt-0.5">•</span>
|
||||
<span>{feedback}</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* 操作按鈕 */}
|
||||
<div className="flex gap-3 justify-center pt-4 border-t">
|
||||
<button
|
||||
onClick={handleRetry}
|
||||
className="px-4 py-2 bg-gray-600 text-white rounded-lg hover:bg-gray-700 transition-colors"
|
||||
>
|
||||
重新錄音
|
||||
</button>
|
||||
<button
|
||||
onClick={handleSubmitResult}
|
||||
className="px-6 py-2 bg-green-600 text-white rounded-lg hover:bg-green-700 transition-colors font-medium"
|
||||
>
|
||||
確認結果
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* 跳過按鈕 */}
|
||||
{!hasAnswered && !isEvaluating && (
|
||||
<div className="text-center">
|
||||
<button
|
||||
onClick={handleSkip}
|
||||
className="px-6 py-2 bg-gray-500 text-white rounded-lg hover:bg-gray-600 transition-colors"
|
||||
>
|
||||
跳過此題
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
|
@ -0,0 +1,400 @@
|
|||
'use client'
|
||||
|
||||
import { useState, useCallback, useRef, useEffect } from 'react'
|
||||
|
||||
export interface AudioRecorderState {
|
||||
isRecording: boolean
|
||||
audioBlob: Blob | null
|
||||
recordingTime: number
|
||||
isProcessing: boolean
|
||||
error: string | null
|
||||
}
|
||||
|
||||
interface AudioRecorderProps {
|
||||
onRecordingComplete: (audioBlob: Blob) => void
|
||||
onRecordingStart?: () => void
|
||||
onRecordingStop?: () => void
|
||||
onError?: (error: string) => void
|
||||
maxDuration?: number // 最大錄音時長（秒）
|
||||
disabled?: boolean
|
||||
className?: string
|
||||
}
|
||||
|
||||
export function AudioRecorder({
|
||||
onRecordingComplete,
|
||||
onRecordingStart,
|
||||
onRecordingStop,
|
||||
onError,
|
||||
maxDuration = 30,
|
||||
disabled = false,
|
||||
className = ""
|
||||
}: AudioRecorderProps) {
|
||||
const [state, setState] = useState<AudioRecorderState>({
|
||||
isRecording: false,
|
||||
audioBlob: null,
|
||||
recordingTime: 0,
|
||||
isProcessing: false,
|
||||
error: null
|
||||
})
|
||||
|
||||
const mediaRecorderRef = useRef<MediaRecorder | null>(null)
|
||||
const streamRef = useRef<MediaStream | null>(null)
|
||||
const audioChunksRef = useRef<Blob[]>([])
|
||||
const timerRef = useRef<NodeJS.Timeout | null>(null)
|
||||
|
||||
// 清理函數
|
||||
const cleanup = useCallback(() => {
|
||||
if (timerRef.current) {
|
||||
clearInterval(timerRef.current)
|
||||
timerRef.current = null
|
||||
}
|
||||
|
||||
if (streamRef.current) {
|
||||
streamRef.current.getTracks().forEach(track => track.stop())
|
||||
streamRef.current = null
|
||||
}
|
||||
|
||||
if (mediaRecorderRef.current && mediaRecorderRef.current.state !== 'inactive') {
|
||||
mediaRecorderRef.current.stop()
|
||||
}
|
||||
|
||||
audioChunksRef.current = []
|
||||
}, [])
|
||||
|
||||
// 組件卸載時清理
|
||||
useEffect(() => {
|
||||
return cleanup
|
||||
}, [cleanup])
|
||||
|
||||
// 開始錄音
|
||||
const startRecording = useCallback(async () => {
|
||||
if (disabled) return
|
||||
|
||||
try {
|
||||
setState(prev => ({ ...prev, error: null, isProcessing: true }))
|
||||
|
||||
// 請求麥克風權限
|
||||
const stream = await navigator.mediaDevices.getUserMedia({
|
||||
audio: {
|
||||
echoCancellation: true,
|
||||
noiseSuppression: true,
|
||||
sampleRate: 16000 // Azure Speech Services 推薦採樣率
|
||||
}
|
||||
})
|
||||
|
||||
streamRef.current = stream
|
||||
audioChunksRef.current = []
|
||||
|
||||
// 創建 MediaRecorder
|
||||
const mediaRecorder = new MediaRecorder(stream, {
|
||||
mimeType: 'audio/webm;codecs=opus' // 現代瀏覽器支援的格式
|
||||
})
|
||||
|
||||
mediaRecorderRef.current = mediaRecorder
|
||||
|
||||
// 處理錄音資料
|
||||
mediaRecorder.ondataavailable = (event) => {
|
||||
if (event.data.size > 0) {
|
||||
audioChunksRef.current.push(event.data)
|
||||
}
|
||||
}
|
||||
|
||||
// 錄音完成處理
|
||||
mediaRecorder.onstop = () => {
|
||||
setState(prev => ({ ...prev, isProcessing: true }))
|
||||
|
||||
const audioBlob = new Blob(audioChunksRef.current, { type: 'audio/webm' })
|
||||
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
audioBlob,
|
||||
isRecording: false,
|
||||
isProcessing: false
|
||||
}))
|
||||
|
||||
onRecordingComplete(audioBlob)
|
||||
cleanup()
|
||||
}
|
||||
|
||||
// 開始錄音
|
||||
mediaRecorder.start(1000) // 每秒收集一次資料
|
||||
|
||||
// 啟動計時器
|
||||
let seconds = 0
|
||||
timerRef.current = setInterval(() => {
|
||||
seconds++
|
||||
setState(prev => ({ ...prev, recordingTime: seconds }))
|
||||
|
||||
// 達到最大時長自動停止
|
||||
if (seconds >= maxDuration) {
|
||||
stopRecording()
|
||||
}
|
||||
}, 1000)
|
||||
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
isRecording: true,
|
||||
recordingTime: 0,
|
||||
isProcessing: false
|
||||
}))
|
||||
|
||||
onRecordingStart?.()
|
||||
|
||||
} catch (error) {
|
||||
const errorMessage = error instanceof Error ? error.message : '麥克風存取失敗'
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
error: errorMessage,
|
||||
isProcessing: false
|
||||
}))
|
||||
onError?.(errorMessage)
|
||||
cleanup()
|
||||
}
|
||||
}, [disabled, maxDuration, onRecordingComplete, onRecordingStart, onError, cleanup])
|
||||
|
||||
// 停止錄音
|
||||
const stopRecording = useCallback(() => {
|
||||
if (mediaRecorderRef.current && mediaRecorderRef.current.state === 'recording') {
|
||||
mediaRecorderRef.current.stop()
|
||||
onRecordingStop?.()
|
||||
}
|
||||
|
||||
if (timerRef.current) {
|
||||
clearInterval(timerRef.current)
|
||||
timerRef.current = null
|
||||
}
|
||||
}, [onRecordingStop])
|
||||
|
||||
// 格式化時間顯示
|
||||
const formatTime = (seconds: number) => {
|
||||
const mins = Math.floor(seconds / 60)
|
||||
const secs = seconds % 60
|
||||
return `${mins.toString().padStart(2, '0')}:${secs.toString().padStart(2, '0')}`
|
||||
}
|
||||
|
||||
return (
|
||||
<div className={`text-center ${className}`}>
|
||||
{/* 錯誤顯示 */}
|
||||
{state.error && (
|
||||
<div className="bg-red-50 border border-red-200 rounded-lg p-3 mb-4">
|
||||
<p className="text-red-700 text-sm">⚠️ {state.error}</p>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* 錄音按鈕 */}
|
||||
<div className="mb-4">
|
||||
{!state.isRecording ? (
|
||||
<button
|
||||
onClick={startRecording}
|
||||
disabled={disabled || state.isProcessing}
|
||||
className={`px-6 py-3 rounded-full font-medium transition-all ${
|
||||
disabled || state.isProcessing
|
||||
? 'bg-gray-300 text-gray-500 cursor-not-allowed'
|
||||
: 'bg-red-500 hover:bg-red-600 text-white transform hover:scale-105'
|
||||
}`}
|
||||
>
|
||||
{state.isProcessing ? (
|
||||
<span className="flex items-center gap-2">
|
||||
<div className="animate-spin w-4 h-4 border-2 border-white border-t-transparent rounded-full"></div>
|
||||
準備中...
|
||||
</span>
|
||||
) : (
|
||||
<span className="flex items-center gap-2">
|
||||
🎤 開始錄音
|
||||
</span>
|
||||
)}
|
||||
</button>
|
||||
) : (
|
||||
<button
|
||||
onClick={stopRecording}
|
||||
className="px-6 py-3 bg-red-600 hover:bg-red-700 text-white rounded-full font-medium animate-pulse transition-all"
|
||||
>
|
||||
<span className="flex items-center gap-2">
|
||||
⏹️ 停止錄音
|
||||
</span>
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* 錄音狀態顯示 */}
|
||||
{state.isRecording && (
|
||||
<div className="bg-red-50 border border-red-200 rounded-lg p-4">
|
||||
<div className="flex items-center justify-center gap-3 mb-2">
|
||||
<div className="w-3 h-3 bg-red-500 rounded-full animate-ping"></div>
|
||||
<span className="text-red-700 font-medium">錄音中</span>
|
||||
</div>
|
||||
|
||||
<div className="text-red-600 text-xl font-mono">
|
||||
{formatTime(state.recordingTime)}
|
||||
</div>
|
||||
|
||||
<div className="text-red-500 text-sm mt-2">
|
||||
最長 {formatTime(maxDuration)}
|
||||
</div>
|
||||
|
||||
{/* 進度條 */}
|
||||
<div className="w-full bg-red-100 rounded-full h-2 mt-3">
|
||||
<div
|
||||
className="bg-red-500 h-2 rounded-full transition-all duration-1000"
|
||||
style={{ width: `${(state.recordingTime / maxDuration) * 100}%` }}
|
||||
></div>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* 錄音完成提示 */}
|
||||
{state.audioBlob && !state.isRecording && (
|
||||
<div className="bg-green-50 border border-green-200 rounded-lg p-3 mt-4">
|
||||
<p className="text-green-700 text-sm">
|
||||
✅ 錄音完成！時長: {formatTime(state.recordingTime)}
|
||||
</p>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* 瀏覽器兼容性提示 */}
|
||||
{!navigator.mediaDevices?.getUserMedia && (
|
||||
<div className="bg-yellow-50 border border-yellow-200 rounded-lg p-3 mt-4">
|
||||
<p className="text-yellow-700 text-sm">
|
||||
⚠️ 您的瀏覽器不支援錄音功能，建議使用 Chrome 或 Firefox
|
||||
</p>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// Hook 版本，提供更靈活的使用方式
|
||||
export function useAudioRecorder(maxDuration: number = 30) {
|
||||
const [state, setState] = useState<AudioRecorderState>({
|
||||
isRecording: false,
|
||||
audioBlob: null,
|
||||
recordingTime: 0,
|
||||
isProcessing: false,
|
||||
error: null
|
||||
})
|
||||
|
||||
const mediaRecorderRef = useRef<MediaRecorder | null>(null)
|
||||
const streamRef = useRef<MediaStream | null>(null)
|
||||
const audioChunksRef = useRef<Blob[]>([])
|
||||
const timerRef = useRef<NodeJS.Timeout | null>(null)
|
||||
|
||||
const cleanup = useCallback(() => {
|
||||
if (timerRef.current) {
|
||||
clearInterval(timerRef.current)
|
||||
timerRef.current = null
|
||||
}
|
||||
|
||||
if (streamRef.current) {
|
||||
streamRef.current.getTracks().forEach(track => track.stop())
|
||||
streamRef.current = null
|
||||
}
|
||||
|
||||
if (mediaRecorderRef.current && mediaRecorderRef.current.state !== 'inactive') {
|
||||
mediaRecorderRef.current.stop()
|
||||
}
|
||||
|
||||
audioChunksRef.current = []
|
||||
}, [])
|
||||
|
||||
const startRecording = useCallback(async () => {
|
||||
try {
|
||||
setState(prev => ({ ...prev, error: null, isProcessing: true }))
|
||||
|
||||
const stream = await navigator.mediaDevices.getUserMedia({
|
||||
audio: {
|
||||
echoCancellation: true,
|
||||
noiseSuppression: true,
|
||||
sampleRate: 16000
|
||||
}
|
||||
})
|
||||
|
||||
streamRef.current = stream
|
||||
audioChunksRef.current = []
|
||||
|
||||
const mediaRecorder = new MediaRecorder(stream, {
|
||||
mimeType: 'audio/webm;codecs=opus'
|
||||
})
|
||||
|
||||
mediaRecorderRef.current = mediaRecorder
|
||||
|
||||
mediaRecorder.ondataavailable = (event) => {
|
||||
if (event.data.size > 0) {
|
||||
audioChunksRef.current.push(event.data)
|
||||
}
|
||||
}
|
||||
|
||||
mediaRecorder.onstop = () => {
|
||||
const audioBlob = new Blob(audioChunksRef.current, { type: 'audio/webm' })
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
audioBlob,
|
||||
isRecording: false,
|
||||
isProcessing: false
|
||||
}))
|
||||
cleanup()
|
||||
}
|
||||
|
||||
mediaRecorder.start(1000)
|
||||
|
||||
let seconds = 0
|
||||
timerRef.current = setInterval(() => {
|
||||
seconds++
|
||||
setState(prev => ({ ...prev, recordingTime: seconds }))
|
||||
|
||||
if (seconds >= maxDuration) {
|
||||
stopRecording()
|
||||
}
|
||||
}, 1000)
|
||||
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
isRecording: true,
|
||||
recordingTime: 0,
|
||||
isProcessing: false
|
||||
}))
|
||||
|
||||
} catch (error) {
|
||||
const errorMessage = error instanceof Error ? error.message : '錄音失敗'
|
||||
setState(prev => ({
|
||||
...prev,
|
||||
error: errorMessage,
|
||||
isProcessing: false
|
||||
}))
|
||||
cleanup()
|
||||
}
|
||||
}, [maxDuration, cleanup])
|
||||
|
||||
const stopRecording = useCallback(() => {
|
||||
if (mediaRecorderRef.current && mediaRecorderRef.current.state === 'recording') {
|
||||
mediaRecorderRef.current.stop()
|
||||
}
|
||||
|
||||
if (timerRef.current) {
|
||||
clearInterval(timerRef.current)
|
||||
timerRef.current = null
|
||||
}
|
||||
}, [])
|
||||
|
||||
const resetRecording = useCallback(() => {
|
||||
cleanup()
|
||||
setState({
|
||||
isRecording: false,
|
||||
audioBlob: null,
|
||||
recordingTime: 0,
|
||||
isProcessing: false,
|
||||
error: null
|
||||
})
|
||||
}, [cleanup])
|
||||
|
||||
// 清理函數
|
||||
useEffect(() => {
|
||||
return cleanup
|
||||
}, [cleanup])
|
||||
|
||||
return {
|
||||
...state,
|
||||
startRecording,
|
||||
stopRecording,
|
||||
resetRecording
|
||||
}
|
||||
}
|
||||
|
|
@ -18,7 +18,7 @@ interface QuizItem {
|
|||
id: string
|
||||
cardId: string
|
||||
cardData: CardState
|
||||
quizType: 'flip-card' | 'vocab-choice'
|
||||
quizType: 'flip-card' | 'vocab-choice' | 'sentence-speaking'
|
||||
order: number
|
||||
isCompleted: boolean
|
||||
wrongCount: number
|
||||
|
|
@ -92,13 +92,20 @@ const generateQuizItemsFromFlashcards = (flashcards: Flashcard[]): QuizItem[] =>
|
|||
const quizItems: QuizItem[] = []
|
||||
let order = 0
|
||||
|
||||
flashcards.forEach((card) => {
|
||||
flashcards.forEach((card, index) => {
|
||||
// 除錯：檢查原始 API 資料
|
||||
if (index === 0) {
|
||||
console.log('🔍 原始 Flashcard 資料 (第一張):', card)
|
||||
}
|
||||
|
||||
// 轉換 Flashcard 為 CardState 格式，確保所有屬性都有值
|
||||
const cardState: CardState = {
|
||||
...card,
|
||||
exampleTranslation: card.exampleTranslation || '', // 確保為 string，不是 undefined
|
||||
updatedAt: card.updatedAt || card.createdAt, // 確保 updatedAt 為 string
|
||||
primaryImageUrl: card.primaryImageUrl || null, // 確保為 null 而非 undefined
|
||||
// 修復圖片 URL - 支援多種屬性名稱
|
||||
primaryImageUrl: card.primaryImageUrl || (card as any).PrimaryImageUrl || null,
|
||||
hasExampleImage: card.hasExampleImage || !!(card.primaryImageUrl || (card as any).PrimaryImageUrl),
|
||||
skipCount: 0,
|
||||
wrongCount: 0,
|
||||
isCompleted: false,
|
||||
|
|
@ -107,7 +114,7 @@ const generateQuizItemsFromFlashcards = (flashcards: Flashcard[]): QuizItem[] =>
|
|||
difficultyLevelNumeric: card.masteryLevel || 1 // 使用 masteryLevel 或預設值 1
|
||||
}
|
||||
|
||||
// 為每張詞卡生成兩種測驗模式
|
||||
// 為每張詞卡生成三種測驗模式
|
||||
quizItems.push(
|
||||
{
|
||||
id: `${card.id}-flip-card`,
|
||||
|
|
@ -128,6 +135,16 @@ const generateQuizItemsFromFlashcards = (flashcards: Flashcard[]): QuizItem[] =>
|
|||
isCompleted: false,
|
||||
wrongCount: 0,
|
||||
skipCount: 0
|
||||
},
|
||||
{
|
||||
id: `${card.id}-sentence-speaking`,
|
||||
cardId: card.id,
|
||||
cardData: cardState,
|
||||
quizType: 'sentence-speaking',
|
||||
order: order++,
|
||||
isCompleted: false,
|
||||
wrongCount: 0,
|
||||
skipCount: 0
|
||||
}
|
||||
)
|
||||
})
|
||||
|
|
|
|||
|
|
@ -35,7 +35,7 @@ export interface CardState extends ApiFlashcard {
|
|||
export interface QuizItem {
|
||||
id: string // 測驗項目ID
|
||||
cardId: string // 所屬詞卡ID
|
||||
quizType: 'flip-card' | 'vocab-choice' // 測驗類型
|
||||
quizType: 'flip-card' | 'vocab-choice' | 'sentence-speaking' // 測驗類型
|
||||
isCompleted: boolean // 個別測驗完成狀態
|
||||
skipCount: number // 跳過次數
|
||||
wrongCount: number // 答錯次數
|
||||
|
|
|
|||
|
|
@ -0,0 +1,192 @@
|
|||
import { API_CONFIG } from '@/lib/config/api'
|
||||
|
||||
export interface PronunciationScores {
|
||||
overall: number
|
||||
accuracy: number
|
||||
fluency: number
|
||||
completeness: number
|
||||
prosody: number
|
||||
}
|
||||
|
||||
export interface WordLevelResult {
|
||||
word: string
|
||||
accuracyScore: number
|
||||
errorType: string
|
||||
suggestion?: string
|
||||
}
|
||||
|
||||
export interface PronunciationResult {
|
||||
assessmentId: string
|
||||
flashcardId: string
|
||||
referenceText: string
|
||||
transcribedText: string
|
||||
scores: PronunciationScores
|
||||
wordLevelResults: WordLevelResult[]
|
||||
feedback: string[]
|
||||
confidenceLevel: number
|
||||
processingTime: number
|
||||
createdAt: string
|
||||
}
|
||||
|
||||
export interface SpeechAssessmentResponse {
|
||||
success: boolean
|
||||
data?: PronunciationResult
|
||||
error?: string
|
||||
message?: string
|
||||
}
|
||||
|
||||
export interface SpeechServiceStatus {
|
||||
isAvailable: boolean
|
||||
serviceName: string
|
||||
checkTime: string
|
||||
message: string
|
||||
}
|
||||
|
||||
class SpeechAssessmentService {
|
||||
private readonly baseUrl: string
|
||||
|
||||
constructor() {
|
||||
this.baseUrl = API_CONFIG.BASE_URL
|
||||
}
|
||||
|
||||
async evaluatePronunciation(
|
||||
audioBlob: Blob,
|
||||
referenceText: string,
|
||||
flashcardId: string,
|
||||
language: string = 'en-US'
|
||||
): Promise<SpeechAssessmentResponse> {
|
||||
try {
|
||||
// 準備 FormData
|
||||
const formData = new FormData()
|
||||
formData.append('audio', audioBlob, 'recording.webm')
|
||||
formData.append('referenceText', referenceText)
|
||||
formData.append('flashcardId', flashcardId)
|
||||
formData.append('language', language)
|
||||
|
||||
console.log('🎤 發送發音評估請求:', {
|
||||
flashcardId,
|
||||
referenceText: referenceText.substring(0, 50) + '...',
|
||||
audioSize: `${(audioBlob.size / 1024).toFixed(1)} KB`,
|
||||
language
|
||||
})
|
||||
|
||||
const response = await fetch(`${this.baseUrl}/api/speech/pronunciation-assessment`, {
|
||||
method: 'POST',
|
||||
body: formData
|
||||
})
|
||||
|
||||
if (!response.ok) {
|
||||
const errorData = await response.json().catch(() => ({}))
|
||||
throw new Error(errorData.message || `HTTP ${response.status}: ${response.statusText}`)
|
||||
}
|
||||
|
||||
const result: SpeechAssessmentResponse = await response.json()
|
||||
|
||||
if (!result.success) {
|
||||
throw new Error(result.message || result.error || '發音評估失敗')
|
||||
}
|
||||
|
||||
console.log('✅ 發音評估完成:', {
|
||||
overallScore: result.data?.scores.overall,
|
||||
confidenceLevel: result.data?.confidenceLevel,
|
||||
processingTime: `${result.data?.processingTime}ms`
|
||||
})
|
||||
|
||||
return result
|
||||
|
||||
} catch (error) {
|
||||
console.error('❌ 發音評估錯誤:', error)
|
||||
|
||||
const errorMessage = error instanceof Error ? error.message : '發音評估失敗'
|
||||
|
||||
return {
|
||||
success: false,
|
||||
error: errorMessage,
|
||||
message: this.getErrorMessage(errorMessage)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async checkServiceStatus(): Promise<SpeechServiceStatus> {
|
||||
try {
|
||||
const response = await fetch(`${this.baseUrl}/api/speech/service-status`)
|
||||
|
||||
if (!response.ok) {
|
||||
throw new Error(`HTTP ${response.status}: ${response.statusText}`)
|
||||
}
|
||||
|
||||
const result = await response.json()
|
||||
|
||||
return result.success ? result.data : {
|
||||
isAvailable: false,
|
||||
serviceName: 'Azure Speech Services',
|
||||
checkTime: new Date().toISOString(),
|
||||
message: '服務狀態檢查失敗'
|
||||
}
|
||||
|
||||
} catch (error) {
|
||||
console.error('語音服務狀態檢查失敗:', error)
|
||||
|
||||
return {
|
||||
isAvailable: false,
|
||||
serviceName: 'Azure Speech Services',
|
||||
checkTime: new Date().toISOString(),
|
||||
message: error instanceof Error ? error.message : '無法連接到語音服務'
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private getErrorMessage(error: string): string {
|
||||
// 將技術錯誤訊息轉換為用戶友善的中文訊息
|
||||
if (error.includes('AUDIO_TOO_SHORT')) {
|
||||
return '錄音時間太短，請至少錄製 1 秒鐘'
|
||||
}
|
||||
if (error.includes('AUDIO_TOO_LARGE')) {
|
||||
return '音頻檔案過大，請縮短錄音時間'
|
||||
}
|
||||
if (error.includes('INVALID_AUDIO_FORMAT')) {
|
||||
return '音頻格式不支援，請重新錄製'
|
||||
}
|
||||
if (error.includes('NO_SPEECH_DETECTED')) {
|
||||
return '未檢測到語音，請確保麥克風正常並大聲說話'
|
||||
}
|
||||
if (error.includes('SPEECH_SERVICE_ERROR')) {
|
||||
return '語音識別服務暫時不可用，請稍後再試'
|
||||
}
|
||||
if (error.includes('NetworkError') || error.includes('fetch')) {
|
||||
return '網路連接錯誤，請檢查網路連接'
|
||||
}
|
||||
|
||||
return '發音評估過程中發生錯誤，請稍後再試'
|
||||
}
|
||||
|
||||
// 輔助方法：檢查瀏覽器是否支援錄音
|
||||
isRecordingSupported(): boolean {
|
||||
return !!(navigator.mediaDevices && navigator.mediaDevices.getUserMedia)
|
||||
}
|
||||
|
||||
// 輔助方法：請求麥克風權限
|
||||
async requestMicrophonePermission(): Promise<boolean> {
|
||||
try {
|
||||
const stream = await navigator.mediaDevices.getUserMedia({ audio: true })
|
||||
// 立即停止，只是檢查權限
|
||||
stream.getTracks().forEach(track => track.stop())
|
||||
return true
|
||||
} catch (error) {
|
||||
console.warn('麥克風權限請求失敗:', error)
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// 導出單例實例
|
||||
export const speechAssessmentService = new SpeechAssessmentService()
|
||||
|
||||
// 導出類型供其他組件使用
|
||||
export type {
|
||||
PronunciationResult,
|
||||
PronunciationScores,
|
||||
WordLevelResult,
|
||||
SpeechAssessmentResponse,
|
||||
SpeechServiceStatus
|
||||
}
|
||||
|
|
@ -623,17 +623,26 @@ graph TD
|
|||
|
||||
## 🚀 實施階段規劃
|
||||
|
||||
### 第一階段：基礎架構
|
||||
1. ✅ 後端 Azure Speech Services 整合
|
||||
2. ✅ 基礎 API 端點實現
|
||||
3. ✅ 資料庫 Schema 更新
|
||||
4. ✅ 環境配置設定
|
||||
### 第一階段：基礎架構 ✅ **已完成**
|
||||
1. ✅ 後端 Azure Speech Services 整合 - Microsoft.CognitiveServices.Speech v1.38.0 安裝完成
|
||||
2. ✅ 基礎 API 端點實現 - SpeechController 完整實現含驗證和錯誤處理
|
||||
3. ✅ 資料庫 Schema 更新 - PronunciationAssessment 實體更新和 Migration 創建
|
||||
4. ✅ 環境配置設定 - AzureSpeechOptions 配置和 appsettings.json 更新
|
||||
5. ✅ 服務依賴注入 - IPronunciationAssessmentService 註冊完成
|
||||
6. ✅ 編譯測試 - 無錯誤，所有組件正常編譯
|
||||
|
||||
### 第二階段：前端整合
|
||||
1. ✅ AudioRecorder 共用組件開發
|
||||
2. ✅ SentenceSpeakingQuiz 組件重構
|
||||
3. ✅ API 服務客戶端實現
|
||||
4. ✅ 複習系統整合
|
||||
**實施詳情**:
|
||||
- **API 端點**: `POST /api/speech/pronunciation-assessment`
|
||||
- **服務狀態端點**: `GET /api/speech/service-status`
|
||||
- **資料模型**: PronunciationResult, PronunciationScores, WordLevelResult
|
||||
- **錯誤處理**: 完整的音頻驗證和 Azure API 錯誤處理
|
||||
- **評分映射**: Azure 評分自動轉換為複習系統信心等級 (0-2分)
|
||||
|
||||
### 第二階段：前端整合 🔄 **進行中**
|
||||
1. ⏳ AudioRecorder 共用組件開發 - 需實現 Web Audio API 錄音功能
|
||||
2. ⏳ SentenceSpeakingQuiz 組件重構 - 基於現有 archive 組件升級
|
||||
3. ⏳ API 服務客戶端實現 - speechAssessmentService.ts 實現
|
||||
4. ⏳ 複習系統整合 - useReviewSession.ts 新增 sentence-speaking quiz type
|
||||
|
||||
### 第三階段：優化和測試
|
||||
1. ✅ 錄音品質優化
|
||||
|
|
@ -742,4 +751,57 @@ graph TD
|
|||
3. ✅ 錯誤提示清晰有幫助
|
||||
4. ✅ 與現有複習流程無縫整合
|
||||
|
||||
這個規格將為 DramaLing 增加強大的口說練習功能，提升學習者的發音能力和語言實際應用技能！
|
||||
---
|
||||
|
||||
## 📈 開發進度更新 (2025-10-08)
|
||||
|
||||
### ✅ 第一階段完成總結
|
||||
|
||||
**完成的檔案和組件**：
|
||||
1. **NuGet 套件**: Microsoft.CognitiveServices.Speech v1.38.0
|
||||
2. **配置類別**: `Models/Configuration/AzureSpeechOptions.cs`
|
||||
3. **DTO 模型**: `Models/DTOs/PronunciationResult.cs`
|
||||
4. **服務介面**: `Contracts/Services/Speech/IPronunciationAssessmentService.cs`
|
||||
5. **核心服務**: `Services/Speech/AzurePronunciationAssessmentService.cs`
|
||||
6. **API 控制器**: `Controllers/SpeechController.cs`
|
||||
7. **資料庫實體**: `Models/Entities/PronunciationAssessment.cs` (更新)
|
||||
8. **資料庫對應**: `Data/DramaLingDbContext.cs` (更新)
|
||||
9. **Migration**: 資料庫結構更新 Migration 已創建
|
||||
10. **依賴注入**: `Extensions/ServiceCollectionExtensions.cs` 服務註冊
|
||||
11. **配置文件**: `appsettings.json` Azure Speech 配置
|
||||
|
||||
**技術驗證**：
|
||||
- ✅ 編譯無錯誤，所有組件正常運作
|
||||
- ✅ Azure Speech SDK 正確整合
|
||||
- ✅ 多維度評分系統實現 (Overall/Accuracy/Fluency/Completeness/Prosody)
|
||||
- ✅ 智能反饋生成邏輯
|
||||
- ✅ 評分映射到複習系統信心等級
|
||||
- ✅ 完整的錯誤處理和驗證
|
||||
|
||||
**下一步開發重點**：
|
||||
1. 🔄 Web Audio API 錄音功能實現
|
||||
2. 🔄 前端 API 客戶端開發
|
||||
3. 🔄 複習系統 quiz type 擴展
|
||||
4. 🔄 前端評分結果 UI 組件
|
||||
|
||||
### 💡 技術亮點
|
||||
|
||||
**智能評分系統**:
|
||||
```csharp
|
||||
private static int MapScoreToConfidence(double overallScore)
|
||||
{
|
||||
return overallScore switch
|
||||
{
|
||||
>= 85 => 2, // 優秀 (高信心)
|
||||
>= 70 => 1, // 良好 (中信心)
|
||||
_ => 0 // 需改善 (低信心)
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
**多維度反饋生成**:
|
||||
- 根據 Azure 評分自動生成中文改善建議
|
||||
- 詞彙級別錯誤識別和具體建議
|
||||
- 流暢度、韻律等多面向評估
|
||||
|
||||
這個規格將為 DramaLing 增加強大的 AI 驅動口說練習功能，提升學習者的發音能力和語言實際應用技能！
|
||||
Loading…
Reference in New Issue