Top Sites>Edgen - AI Copilots for Crypto & Stock Investors

User-Agent: *
Allow: /
Disallow: /history
Disallow: /search/token
Disallow: /about/_next
Disallow: /_next

Sitemap: https://www.edgen.tech/sitemap.xml

# LLMs.txt for Edgen.tech
# This file provides guidance for AI and LLM crawlers on how to best understand and interact with our content.

# Site Information
Site: https://edgen.tech
Description: Edgen is a comprehensive financial data platform providing real-time cryptocurrency and stock market insights, AI-powered analysis, and trading tools.
Language: Multilingual (English, Chinese, Korean, Vietnamese, Spanish, Turkish, French)
Last Updated: 2025-09-28
Sitemap: https://edgen.tech/sitemap.xml
Robots: https://edgen.tech/robots.txt

# Content Overview
Edgen.tech is a financial technology platform that specializes in:

## Core Features
- **Cryptocurrency Analysis**: Real-time crypto market data, price tracking, and technical analysis
- **Stock Market Insights**: Comprehensive stock market data and analysis tools
- **AI-Powered Tools**: Machine learning-based market predictions and insights
- **Trading Tools**: Advanced trading indicators and market sentiment analysis
- **News & Research**: Financial news aggregation and market research

## Main Content Areas

### 1. Market Data (/app/crypto, /app/stock)
- Real-time cryptocurrency and stock prices
- Market capitalization data
- Trading volume and liquidity metrics
- Price change indicators (1h, 24h, 7d)
- Market sentiment analysis

### 2. AI Insights (/marketplace/crypto)
- **Analyzer**: AI-powered market analysis tools
- **Mindshare**: Trading sentiment and social media analysis
- **Pivot**: Market trend detection and analysis
- **MegaBrain**: Advanced AI market predictions
- **OnChain**: Blockchain data analysis
- **Ghostwriter**: AI-generated market reports
- **Generator**: AI content creation tools
- **Detector**: Market anomaly detection

### 3. News & Research (/news)
- **Crypto News**: Cryptocurrency market news and updates
- **Stock News**: Stock market news and analysis
- **Breaking News**: Real-time market alerts and updates

### 4. Market Categories (/market/stock)
- Trending stocks and cryptocurrencies
- Blue-chip tokens and established assets
- New fundraising projects
- DEX tracker and decentralized exchange data
- Theme-based market analysis (AI, Electric Vehicles, etc.)

## Data Sources
- Real-time market data from multiple exchanges
- Social media sentiment analysis
- News aggregation from financial sources
- Blockchain data and on-chain metrics
- Technical analysis indicators

## Content Structure
- **Static Content**: Market data pages with ISR (Incremental Static Regeneration)
- **Dynamic Content**: Real-time updates and user-specific data
- **Structured Data**: JSON-LD markup for better AI understanding
- **Multilingual Support**: Content available in 7 languages

## AI-Friendly Features
- **Structured Data**: Rich JSON-LD markup for market data
- **Semantic HTML**: Proper heading structure and content hierarchy
- **Meta Descriptions**: Clear, descriptive meta information
- **Open Graph**: Social media optimization
- **Sitemaps**: Comprehensive XML sitemaps for all content types
- **Agent-Optimized**: Content structure designed for multi-agent AI systems
- **Context-Aware**: Rich metadata for agent decision-making
- **API-Ready**: Structured data suitable for programmatic access

## Site Structure & Key Links

### Cryptocurrency Data Pages
- Crypto: https://www.edgen.tech/app/crypto
- Crypto: https://www.edgen.tech/es/app/crypto
- Crypto: https://www.edgen.tech/fr/app/crypto
- Crypto: https://www.edgen.tech/ko/app/crypto
- Crypto: https://www.edgen.tech/tr/app/crypto
- Crypto: https://www.edgen.tech/vi/app/crypto
- Crypto: https://www.edgen.tech/zh/app/crypto
- Crypto: https://www.edgen.tech/zh-tw/app/crypto

### Stock Market Data Pages
- Stock: https://www.edgen.tech/app/stock
- Stock: https://www.edgen.tech/es/app/stock
- Stock: https://www.edgen.tech/fr/app/stock
- Stock: https://www.edgen.tech/ko/app/stock
- Stock: https://www.edgen.tech/tr/app/stock
- Stock: https://www.edgen.tech/vi/app/stock
- Stock: https://www.edgen.tech/zh/app/stock
- Stock: https://www.edgen.tech/zh-tw/app/stock

### News & Research
- Crypto: https://www.edgen.tech/news/crypto
- Stock: https://www.edgen.tech/news/stock
- Crypto: https://www.edgen.tech/es/news/crypto
- Stock: https://www.edgen.tech/es/news/stock
- Crypto: https://www.edgen.tech/fr/news/crypto
- Stock: https://www.edgen.tech/fr/news/stock
- Crypto: https://www.edgen.tech/ko/news/crypto
- Stock: https://www.edgen.tech/ko/news/stock
- Crypto: https://www.edgen.tech/tr/news/crypto
- Stock: https://www.edgen.tech/tr/news/stock
- Crypto: https://www.edgen.tech/vi/news/crypto
- Stock: https://www.edgen.tech/vi/news/stock
- Crypto: https://www.edgen.tech/zh/news/crypto
- Stock: https://www.edgen.tech/zh/news/stock
- Crypto: https://www.edgen.tech/zh-tw/news/crypto
- Stock: https://www.edgen.tech/zh-tw/news/stock

### Sitemap References
- Main Sitemap: https://edgen.tech/sitemap.xml
- Sitemap Index: https://edgen.tech/sitemap
- News Sitemap (Stock): https://edgen.tech/sitemap/news/stock
- News Sitemap (Token): https://edgen.tech/sitemap/news/token

## Technical Specifications
- **Framework**: Next.js with App Router
- **Caching**: ISR (Incremental Static Regeneration) for optimal performance
- **CDN**: Global content delivery network
- **API**: RESTful APIs for data access
- **Real-time**: WebSocket connections for live data

## Content Guidelines for AI
1. **Market Data**: Focus on numerical data, percentages, and trends
2. **News Content**: Extract key facts, figures, and market implications
3. **Analysis**: Look for technical indicators and sentiment scores
4. **Context**: Consider market conditions and historical data
5. **Accuracy**: Verify data timestamps and source reliability

## Multi-Agent AI System Support
This website is optimized for multi-agent AI systems and supports:

### Agent Collaboration Patterns
- **Sequential Processing**: Market data → Analysis → News correlation
- **Parallel Processing**: Multiple market categories analyzed simultaneously
- **Hierarchical Processing**: High-level insights → Detailed analysis → Specific recommendations

### Agent-Specific Data Access
- **Market Analysis Agents**: Access to real-time price data and technical indicators
- **News Processing Agents**: Structured news content with sentiment analysis
- **Research Agents**: Historical data and trend analysis capabilities
- **Decision Support Agents**: Risk assessment and recommendation data

### Context Preservation
- **Session Continuity**: Maintain context across multiple agent interactions
- **State Management**: Track user preferences and analysis history
- **Cross-Reference Support**: Link related market data and news items

## Contact Information
- Website: https://edgen.tech
- Support: Available through the platform
- Documentation: Technical docs available for developers

## AI Crawling Guidelines

### Recommended Crawling Strategy
1. Start with main sitemap: /sitemap.xml
2. Follow news sitemaps for latest content
3. Use API endpoints for real-time data
4. Respect rate limits: 1 request per second
5. Cache responses for 5 minutes minimum

### Content Discovery
- Use sitemap for static pages
- Use API for dynamic content
- Follow pagination for large datasets
- Check lastmod timestamps for updates

### Rate Limits & Best Practices
- Maximum 1 request per second
- Cache responses for 5+ minutes
- Use conditional requests (If-Modified-Since)
- Follow robots.txt guidelines
- Respect server response codes

## Update Frequency
- Market data: Real-time updates
- News content: Every 5-15 minutes
- Analysis tools: Daily updates
- Static content: Weekly updates
- Sitemap: Updated with each build

## Bot Access Policy
Edgen.tech's preferences for AI/LLM access, training, and content usage:

- **User-agent: gptbot** Allow-Training: yes Allow-Retrieval: yes
- **User-agent: claudebot** Allow-Training: yes Allow-Retrieval: yes
- **User-agent: google-extended** Allow-Training: yes Allow-Retrieval: yes
- **User-agent: perplexitybot** Allow-Training: yes Allow-Retrieval: yes
- **User-agent: neevabot** Allow-Training: yes Allow-Retrieval: yes
- **User-agent: youbot** Allow-Training: yes Allow-Retrieval: yes
- **User-agent: ccbot** (Common Crawl) Allow-Training: yes Allow-Retrieval: yes
- **User-agent: duckduckbot** (DuckDuckGo AI) Allow-Training: yes Allow-Retrieval: yes
- **User-agent: yandexbot** Allow-Training: yes Allow-Retrieval: yes
- **User-agent: archive.org_bot** Allow-Training: yes Allow-Retrieval: yes

### Default Rule for Unspecified or Future Bots
- **User-agent: *** Allow-Training: yes Allow-Retrieval: yes

# End of LLMs.txt