Building a Real-time NSE Index Analysis Dashboard with Streamlit and Python

In this technical blog post, we’ll explore how to build a real-time dashboard for analyzing National Stock Exchange (NSE) indices using Python, Streamlit, and various data visualization libraries. We’ll cover everything from setting up the environment to implementing interactive visualizations and handling real-time data updates.

Project Overview

The NSE Index Analysis Dashboard is a web application that provides real-time analysis and visualization of Indian stock market indices. It fetches live data from the NSE API and presents it through an intuitive interface with interactive charts, filters, and detailed analysis tools.

Key Features:

• Real-time data fetching from NSE API
• Interactive data visualizations
• Multiple analysis views (Overview, Performance, Top Movers, etc.)
• Custom filtering and search capabilities
• Downloadable data in CSV format

Technical Stack

The project uses the following technologies:

streamlit        # Web application framework
pandas          # Data manipulation and analysis
requests        # HTTP library for API calls
plotly          # Interactive visualization library
numpy           # Numerical computing library

Implementation Details

1. Data Fetching and Authentication

One of the crucial aspects of the application is fetching data from the NSE API. The implementation requires careful handling of sessions and headers:

@st.cache_data(ttl=300)  # Cache for 5 minutes
def fetch_nse_data():
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36',
        'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
        'Accept-Language': 'en-US,en;q=0.5',
        'Referer': 'https://www.nseindia.com/market-data/live-market-indices'
    }

    session = requests.Session()

    # Essential steps for proper authentication
    session.get('https://www.nseindia.com/', headers=headers)
    session.get('https://www.nseindia.com/market-data/live-market-indices', headers=headers)

    response = session.get('https://www.nseindia.com/api/allIndices', headers=headers)
    # Process response...

2. Data Processing Pipeline

The data processing pipeline involves several steps:

1. Fetch Raw Data: Get JSON response from NSE API
2. Data Transformation: Convert to Pandas DataFrame
3. Data Cleaning: Handle missing values and format columns
4. Numeric Processing: Convert string values to proper numeric format

3. Interactive Visualizations

The dashboard uses Plotly for creating interactive visualizations. Here’s an example of creating a market sentiment visualization:

def create_sentiment_chart(filtered_df):
    positive_count = len(filtered_df[filtered_df['percentChange'] > 0])
    negative_count = len(filtered_df[filtered_df['percentChange'] < 0])
    neutral_count = len(filtered_df[filtered_df['percentChange'] == 0])

    sentiment_data = pd.DataFrame({
        'Sentiment': ['Positive', 'Negative', 'Neutral'],
        'Count': [positive_count, negative_count, neutral_count]
    })

    fig = px.pie(
        sentiment_data, 
        values='Count', 
        names='Sentiment',
        title="Market Sentiment Distribution",
        color_discrete_map={'Positive': 'green', 'Negative': 'red', 'Neutral': 'gray'}
    )
    return fig

Key Features

1. Real-time Updates

• Data is cached for 5 minutes using @st.cache_data
• Automatic refresh of visualizations
• Status indicators for data freshness

2. Interactive Filtering

• Category-based filtering (NIFTY, Sectoral, Bond, etc.)
• Text-based search functionality
• Column selection for custom views

3. Multi-view Analysis

Overview Tab

• Top 10 indices by value
• Daily change distribution
• Market sentiment analysis

Performance Tab

• Time period comparison
• Volatility analysis
• Historical performance tracking

Top Movers Tab

• Gainers and losers analysis
• Sector performance heatmap

Challenges and Solutions

1. API Authentication

Challenge: NSE API requires specific headers and session management.
Solution: Implemented a robust session management system with proper headers and cookie handling.

2. Data Refresh

Challenge: Balancing real-time updates with API rate limits.
Solution: Used Streamlit’s caching mechanism with a 5-minute TTL.

3. Performance Optimization

Challenge: Handling large datasets without impacting UI responsiveness.
Solution:

• Implemented efficient data filtering
• Used lazy loading for charts
• Optimized DataFrame operations

Code Walkthrough

1. Application Structure

nse_streamlit_dashboard/
├── nse_dashboard.py     # Main application file
├── requirements.txt     # Dependencies
└── README.md           # Documentation

2. Key Components

Data Fetching

@st.cache_data(ttl=300)
def fetch_nse_data():
    # Implementation details...

Visualization Components

def create_performance_chart(data):
    fig = go.Figure()
    # Add traces and styling...
    return fig

Interactive Elements

# Sidebar filters
index_categories = st.sidebar.selectbox(
    "Select Index Category",
    ["All", "NIFTY", "Sectoral", "Thematic", "Bond", "Broad Market"]
)

Conclusion

This NSE Index Analysis Dashboard demonstrates the power of combining Python’s data analysis capabilities with Streamlit’s interactive web framework. The application provides a robust solution for real-time market analysis with a focus on user experience and performance.

Future Enhancements

1. Add technical indicators and analysis
2. Implement historical data analysis
3. Add export capabilities for charts
4. Integrate with other financial APIs
5. Add user authentication and personalization

---

The complete source code for this project is available on GitHub and also deployed on Streamlit Community Cloud – https://nse-dashboard.streamlit.app