主页

索引

模块索引

搜索页面

6.3.2. Transformers

• Transformers
  • 简介
  • GET STARTED
    • Quick tour
      • Pipeline
      • AutoClass
      • Custom model builds
      • Trainer
    • Installation
    • 环境变量
      • Fetch models and tokenizers to use offline
        • Use the from_pretrained() and save_pretrained() workflow
        • Programmatically download files with the huggingface_hub library
  • TUTORIALS
    • Pipelines for inference
    • Load pretrained instances with an AutoClass
      • AutoTokenizer
      • AutoImageProcessor
      • AutoFeatureExtractor
      • AutoProcessor
      • AutoModel
    • Preprocess data
      • Natural Language Processing
        • Pad
        • Truncation
        • Build tensors
      • Audio
        • Pading
      • Computer vision
        • image augmentation
        • Pading
    • Fine-tune a pretrained model
      • Train with PyTorch Trainer
        • Training hyperparameters
        • Evaluate
        • Trainer
      • Train in native PyTorch
        • DataLoader
        • Optimizer and learning rate scheduler
        • Training loop
        • Evaluate
    • Train with a script
    • Distributed training with Accelerate
      • Backward
    • Transformers Agent
      • 示例
      • Quickstart
        • Single execution (run)
        • Chat-based execution (chat)
      • 原理
        • Agents
        • Tools
      • Resource
        • A curated set of tools
        • Custom tools
      • Code generation
      • Practice
  • TASK GUIDES
    • NATURAL LANGUAGE PROCESSING
    • AUDIO
    • COMPUTER VISION
    • MULTIMODAL
  • DEVELOPER GUIDES
    • Transformers Notebooks with examples
    • Community resources
  • PERFORMANCE AND SCALABILITY
  • CONCEPTUAL GUIDES
    • Philosophy
      • Main concepts
    • Glossary
      • feed forward chunking
      • image patch
    • How Transformers solve tasks
    • The Transformer model family
      • Computer vision
      • Natural language processing
      • Audio
      • Multimodal
      • Reinforcement learning
    • Summary of the tokenizers
    • Padding and truncation
    • Model training anatomy
      • Anatomy of Model’s Operations
      • Anatomy of Model’s Memory
  • API
    • MAIN CLASSES
      • Agents
      • Auto Classes
      • Callbacks
      • Logging
• Transformers 4.45.2
  • Tutorials
    • Load pretrained instances with an AutoClass
      • AutoBackbone
    • Generation with LLMs
      • Generate text
      • Common pitfalls
        • Generated output is too short/long
        • Incorrect generation mode
        • Wrong padding side
        • Wrong prompt
    • Chatting with Transformers
      • Choosing a chat model
      • 相关代码
      • Performance, memory and hardware
        • Memory considerations
        • Performance considerations
  • TASK GUIDES
    • COMPUTER VISION
      • Image-to-Image
      • Image Feature Extraction
      • Mask Generation
      • Keypoint Detection
      • Knowledge Distillation for Computer Vision
    • MULTIMODAL
    • Generation
      • Text generation strategies
        • Default text generation configuration
        • Customize text generation
        • Save a custom decoding strategy with your model
        • Streaming
        • Watermarking
        • Decoding strategies
          • Greedy Search
          • Contrastive search
          • Multinomial sampling
          • Beam-search decoding
          • Beam-search multinomial sampling
          • Diverse beam search decoding
          • Speculative Decoding
          • Universal Assisted Decoding
          • DoLa Decoding
      • Best Practices for Generation with Cache
        • What is Cache and why we should care
        • Generate with Cache
          • Quantized Cache
          • Offloaded Cache
          • Static Cache
          • Offloaded Static Cache
          • Sliding Window Cache
          • Sink Cache
          • Encoder-Decoder Cache
        • Model-specific Cache Classes
    • Prompting
      • Image tasks with IDEFICS
      • LLM prompting guide
  • Developer guides
    • Use fast tokenizers from 🤗 Tokenizers
      • Loading directly from the tokenizer object
      • Loading from a JSON file
    • Use model-specific APIs
      • Configuration
      • Model
        • Model heads
      • Tokenizer
      • Image processor
      • Backbone
      • Feature extractor
      • Processor
      • Building custom models
    • Chat Templates
      • Introduce
      • How do I use chat templates
      • 核心参数
        • add_generation_prompt 参数
        • continue_final_message 参数
        • tokenize 参数
      • Advanced: How do chat templates work?
      • Advanced: Adding and editing chat templates
        • How do I create a chat template?
      • Advanced: Template writing tips
        • Trimming whitespace
        • Callable functions
        • Compatibility with non-Python Jinja
    • Trainer
    • Export to ONNX
      • Exporting a 🤗 Transformers model to ONNX with CLI
      • Exporting a 🤗 Transformers model to ONNX with optimum.onnxruntime
      • Exporting a model with transformers.onnx
    • Interoperability with GGUF files
  • Quantization Methods
    • Quantization
    • bitsandbytes
    • GPTQ
      • ExLlama
    • AWQ
    • AQLM
    • Quanto
    • EETQ
    • HQQ
    • FBGEMM FP8
    • Optimum
    • TorchAO
    • Compressed Tensors
  • Performance and scalability
    • LLM inference optimization
      • Static kv-cache and torch.compile
      • Speculative decoding
        • Prompt lookup decoding
      • Attention optimizations
        • FlashAttention-2
        • PyTorch scaled dot product attention
      • Quantization
    • Efficient training techniques
      • Methods and tools for efficient training on a single GPU
        • Batch size choice
        • Gradient Accumulation
        • Gradient Checkpointing
        • Mixed precision training
          • fp16
          • BF16
          • TF32
        • Flash Attention 2
        • Optimizer choice
        • Data preloading
        • DeepSpeed ZeRO
        • Using torch.compile
        • Using 🤗 PEFT
        • Using 🤗 Accelerate
      • Multiple GPUs and parallelism
      • Fully Sharded Data Parallel
      • DeepSpeed
      • Efficient Training on CPU
      • Distributed CPU training
    • Optimizing inference
      • CPU inference
        • 🤗 Optimum
      • GPU inference
        • FlashAttention-2
        • BetterTransformer
        • bitsandbytes
    • Others
      • Optimize inference using torch.compile()
  • Conceptual guides
    • Glossary
      • DataParallel (DP)
      • PipelineParallel (PP)
      • Tensor Parallelism (TP)
      • Tensor Parallelism vs. Data Parallelism vs. Pipeline Parallelism
      • Zero Redundancy Optimizer (ZeRO)
      • Fully Sharded Data Parallel
    • Optimizing LLMs for Speed and Memory
      • 1. Lower Precision
      • 2. Flash Attention
      • 3. Architectural Innovations
        • 3.1 Improving positional embeddings of LLMs
        • 3.2 The key-value cache
          • 3.2.1 Multi-round conversation
          • 3.2.2 Multi-Query-Attention (MQA)
          • 3.2.3 Grouped-Query-Attention (GQA)
  • API
    • Main Classes
      • Auto Classes
      • Backbone
      • Callbacks
      • Configuration
      • Data Collator
      • Logging
      • Models
      • Text Generation
      • ONNX
      • Optimization
      • Model outputs
      • Pipelines
      • Processors
        • Multi-modal processors
      • Quantization
      • Tokenizer
      • Trainer
    • Models
      • Text models
        • Qwen2
        • Qwen2_VL
        • CPM
        • DPR
  • 其他-fromGPT

主页

索引

模块索引

搜索页面